{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Y5yTxMNOKnOC",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Copyright 2024 Google LLC\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "R5Hp34AVK7ba"
   },
   "source": [
    "## Overview\n",
    "\n",
    "This tutorial demonstrates how to use the Vertex AI SDK for Python to set up Vertex AI Model Monitoring V2 for your model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DyXl559OtyK_"
   },
   "source": [
    "### Objective\n",
    "\n",
    "In this tutorial, you'll complete the following steps:\n",
    "\n",
    "- Upload a custom model to Vertex AI Model Registry.\n",
    "- Create a model monitor.\n",
    "- Create Vertex AI batch prediction job.\n",
    "- Run an on-demand model monitoring job to analyze data drift between the batch prediction job results and the training dataset.\n",
    "- Create another Vertex AI batch prediction job.\n",
    "- Run an on-demand model monitoring job to analyze data drift between the batch prediction job results and the previous batch prediction job.\n",
    "- Run an on-demand model monitoring job to analyze the feature attribution drift between the batch prediction job results and a baseline dataset in Google Cloud Storage.\n",
    "\n",
    "\n",
    "### Costs\n",
    "\n",
    "Vertex AI Model Monitoring v2 is free during the public preview period, but you will still be billed for the following Google Cloud services:\n",
    "\n",
    "* [BigQuery](https://cloud.google.com/bigquery/pricing)\n",
    "* [Cloud Storage](https://cloud.google.com/storage/pricing)\n",
    "* [Vertex AI Online Prediction](https://cloud.google.com/vertex-ai/pricing#prediction-prices)\n",
    "* [Vertex AI Batch Explanation Job](https://cloud.google.com/vertex-ai/pricing#prediction-prices) (if you run the feature attribution drift example)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oYmaoz9MNLHd"
   },
   "source": [
    "## Getting Started"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NvW5zNXmNMwm"
   },
   "source": [
    "### Install Vertex AI SDK and other required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5FakhYktNO7Y",
    "outputId": "1d2e53b8-eb15-4012-b15f-d2dc33b08d89",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: google-cloud-bigquery in /opt/conda/lib/python3.10/site-packages (3.38.0)\n",
      "Collecting pandas==1.5.3\n",
      "  Downloading pandas-1.5.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\n",
      "Requirement already satisfied: pandas-gbq in /opt/conda/lib/python3.10/site-packages (0.29.2)\n",
      "Requirement already satisfied: pyarrow in /opt/conda/lib/python3.10/site-packages (21.0.0)\n",
      "Collecting tensorflow-data-validation\n",
      "  Using cached tensorflow_data_validation-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)\n",
      "Requirement already satisfied: google-cloud-aiplatform in /opt/conda/lib/python3.10/site-packages (1.120.0)\n",
      "Collecting tensorflow-metadata\n",
      "  Downloading tensorflow_metadata-1.17.2-py3-none-any.whl.metadata (2.5 kB)\n",
      "Requirement already satisfied: google-api-core in /opt/conda/lib/python3.10/site-packages (2.26.0)\n",
      "Requirement already satisfied: google-auth in /opt/conda/lib/python3.10/site-packages (2.41.1)\n",
      "Requirement already satisfied: google-cloud-core in /opt/conda/lib/python3.10/site-packages (2.4.3)\n",
      "Requirement already satisfied: google-resumable-media in /opt/conda/lib/python3.10/site-packages (2.7.2)\n",
      "Collecting tfx-bsl\n",
      "  Downloading tfx_bsl-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/lib/python3.10/site-packages (from pandas==1.5.3) (2.9.0.post0)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas==1.5.3) (2025.2)\n",
      "Requirement already satisfied: numpy>=1.21.0 in /opt/conda/lib/python3.10/site-packages (from pandas==1.5.3) (1.26.4)\n",
      "Requirement already satisfied: packaging>=24.2.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-bigquery) (25.0)\n",
      "Requirement already satisfied: requests<3.0.0,>=2.21.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-bigquery) (2.32.5)\n",
      "Requirement already satisfied: googleapis-common-protos<2.0.0,>=1.56.2 in /opt/conda/lib/python3.10/site-packages (from google-api-core) (1.70.0)\n",
      "Requirement already satisfied: protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.19.5 in /opt/conda/lib/python3.10/site-packages (from google-api-core) (6.31.1)\n",
      "Requirement already satisfied: proto-plus<2.0.0,>=1.22.3 in /opt/conda/lib/python3.10/site-packages (from google-api-core) (1.26.1)\n",
      "Requirement already satisfied: cachetools<7.0,>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from google-auth) (6.2.0)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.10/site-packages (from google-auth) (0.4.2)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.10/site-packages (from google-auth) (4.9.1)\n",
      "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /opt/conda/lib/python3.10/site-packages (from google-resumable-media) (1.7.1)\n",
      "Requirement already satisfied: grpcio<2.0.0,>=1.33.2 in /opt/conda/lib/python3.10/site-packages (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery) (1.75.1)\n",
      "Requirement already satisfied: grpcio-status<2.0.0,>=1.33.2 in /opt/conda/lib/python3.10/site-packages (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery) (1.75.1)\n",
      "Requirement already satisfied: typing-extensions~=4.12 in /opt/conda/lib/python3.10/site-packages (from grpcio<2.0.0,>=1.33.2->google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery) (4.15.0)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas==1.5.3) (1.17.0)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests<3.0.0,>=2.21.0->google-cloud-bigquery) (3.4.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests<3.0.0,>=2.21.0->google-cloud-bigquery) (3.10)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests<3.0.0,>=2.21.0->google-cloud-bigquery) (2.5.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests<3.0.0,>=2.21.0->google-cloud-bigquery) (2025.10.5)\n",
      "Requirement already satisfied: pyasn1>=0.1.3 in /opt/conda/lib/python3.10/site-packages (from rsa<5,>=3.1.4->google-auth) (0.6.1)\n",
      "Requirement already satisfied: setuptools in /opt/conda/lib/python3.10/site-packages (from pandas-gbq) (80.9.0)\n",
      "Requirement already satisfied: db-dtypes<2.0.0,>=1.0.4 in /opt/conda/lib/python3.10/site-packages (from pandas-gbq) (1.4.3)\n",
      "Requirement already satisfied: pydata-google-auth>=1.5.0 in /opt/conda/lib/python3.10/site-packages (from pandas-gbq) (1.9.1)\n",
      "Requirement already satisfied: google-auth-oauthlib>=0.7.0 in /opt/conda/lib/python3.10/site-packages (from pandas-gbq) (1.2.2)\n",
      "Collecting absl-py<2.0.0,>=0.9 (from tensorflow-data-validation)\n",
      "  Downloading absl_py-1.4.0-py3-none-any.whl.metadata (2.3 kB)\n",
      "Requirement already satisfied: joblib>=1.2.0 in /opt/conda/lib/python3.10/site-packages (from tensorflow-data-validation) (1.5.2)\n",
      "INFO: pip is looking at multiple versions of tensorflow-data-validation to determine which version is compatible with other requirements. This could take a while.\n",
      "Collecting tensorflow-data-validation\n",
      "  Using cached tensorflow_data_validation-1.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)\n",
      "  Using cached tensorflow_data_validation-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\n",
      "  Using cached tensorflow_data_validation-1.15.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\n",
      "  Downloading tensorflow_data_validation-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\n",
      "Collecting apache-beam<3,>=2.47 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading apache_beam-2.68.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)\n",
      "Collecting db-dtypes<2.0.0,>=1.0.4 (from pandas-gbq)\n",
      "  Using cached db_dtypes-1.4.3-py3-none-any.whl.metadata (3.0 kB)\n",
      "INFO: pip is still looking at multiple versions of tensorflow-data-validation to determine which version is compatible with other requirements. This could take a while.\n",
      "  Using cached db_dtypes-1.4.2-py2.py3-none-any.whl.metadata (3.0 kB)\n",
      "Collecting pyarrow\n",
      "  Downloading pyarrow-10.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)\n",
      "Collecting pyfarmhash<0.4,>=0.2.2 (from tensorflow-data-validation)\n",
      "  Using cached pyfarmhash-0.3.2-cp310-cp310-linux_x86_64.whl\n",
      "Collecting tensorflow<2.17,>=2.16 (from tensorflow-data-validation)\n",
      "  Downloading tensorflow-2.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)\n",
      "Collecting tensorflow-metadata\n",
      "  Downloading tensorflow_metadata-1.16.1-py3-none-any.whl.metadata (2.4 kB)\n",
      "Collecting tensorflow<2.16,>=2.15 (from tensorflow-data-validation)\n",
      "  Downloading tensorflow-2.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)\n",
      "Collecting tensorflow-metadata\n",
      "  Downloading tensorflow_metadata-1.15.0-py3-none-any.whl.metadata (2.4 kB)\n",
      "Collecting tfx-bsl\n",
      "  Downloading tfx_bsl-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\n",
      "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
      "Collecting tensorflow~=2.15 (from tensorflow-data-validation)\n",
      "  Downloading tensorflow-2.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.5 kB)\n",
      "Collecting pandas-gbq\n",
      "  Using cached pandas_gbq-0.29.2-py3-none-any.whl.metadata (3.6 kB)\n",
      "Collecting proto-plus<2.0.0,>=1.22.3 (from google-api-core)\n",
      "  Downloading proto_plus-1.26.1-py3-none-any.whl.metadata (2.2 kB)\n",
      "  Downloading proto_plus-1.26.0-py3-none-any.whl.metadata (2.2 kB)\n",
      "INFO: pip is looking at multiple versions of proto-plus to determine which version is compatible with other requirements. This could take a while.\n",
      "  Downloading proto_plus-1.25.0-py3-none-any.whl.metadata (2.2 kB)\n",
      "  Downloading proto_plus-1.24.0-py3-none-any.whl.metadata (2.2 kB)\n",
      "  Downloading proto_plus-1.23.0-py3-none-any.whl.metadata (2.2 kB)\n",
      "  Downloading proto_plus-1.22.3-py3-none-any.whl.metadata (2.2 kB)\n",
      "Collecting grpcio-status<2.0.0,>=1.33.2 (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery)\n",
      "  Downloading grpcio_status-1.75.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "INFO: pip is still looking at multiple versions of proto-plus to determine which version is compatible with other requirements. This could take a while.\n",
      "  Downloading grpcio_status-1.75.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
      "  Downloading grpcio_status-1.74.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.73.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.73.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.72.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.72.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.71.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.19.5 (from google-api-core)\n",
      "  Downloading protobuf-5.29.5-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes)\n",
      "Collecting grpcio-status<2.0.0,>=1.33.2 (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery)\n",
      "  Downloading grpcio_status-1.71.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.70.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.69.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.68.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.68.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.67.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.67.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.66.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.66.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.66.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.65.5-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.65.4-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.65.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.65.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.64.3-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.64.1-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.64.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.63.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.63.0-py3-none-any.whl.metadata (1.1 kB)\n",
      "  Downloading grpcio_status-1.62.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.19.5 (from google-api-core)\n",
      "  Downloading protobuf-4.25.8-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)\n",
      "INFO: pip is looking at multiple versions of tensorflow-metadata to determine which version is compatible with other requirements. This could take a while.\n",
      "Collecting tensorflow-metadata\n",
      "  Downloading tensorflow_metadata-1.16.0-py3-none-any.whl.metadata (2.4 kB)\n",
      "  Downloading tensorflow_metadata-1.14.0-py3-none-any.whl.metadata (2.1 kB)\n",
      "Collecting tfx-bsl\n",
      "  Downloading tfx_bsl-1.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (10 kB)\n",
      "INFO: pip is still looking at multiple versions of tensorflow-metadata to determine which version is compatible with other requirements. This could take a while.\n",
      "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
      "Collecting grpcio-status<2.0.0,>=1.33.2 (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery)\n",
      "  Downloading grpcio_status-1.62.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.62.1-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.62.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.61.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.60.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.60.1-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.60.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.59.5-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.59.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.59.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.58.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)\n",
      "  Downloading grpcio_status-1.56.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.56.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.55.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.54.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.54.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.54.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.53.2-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.53.1-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.53.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.51.3-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.51.1-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.50.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.49.1-py3-none-any.whl.metadata (1.3 kB)\n",
      "  Downloading grpcio_status-1.48.2-py3-none-any.whl.metadata (1.2 kB)\n",
      "Collecting protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<7.0.0,>=3.19.5 (from google-api-core)\n",
      "  Using cached protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (679 bytes)\n",
      "Collecting google-api-python-client<2,>=1.7.11 (from tfx-bsl)\n",
      "  Downloading google_api_python_client-1.12.11-py2.py3-none-any.whl.metadata (4.2 kB)\n",
      "Collecting tensorflow-serving-api<3,>=2.13.0 (from tfx-bsl)\n",
      "  Downloading tensorflow_serving_api-2.19.1-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "Collecting crcmod<2.0,>=1.7 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading crcmod-1.7.tar.gz (89 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting orjson<4,>=3.9.7 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading orjson-3.11.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (41 kB)\n",
      "Collecting dill<0.3.2,>=0.3.1.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading dill-0.3.1.1.tar.gz (151 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting fastavro<2,>=0.23.6 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading fastavro-1.12.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (5.8 kB)\n",
      "Collecting fasteners<1.0,>=0.3 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading fasteners-0.20-py3-none-any.whl.metadata (4.8 kB)\n",
      "Collecting grpcio<2.0.0,>=1.33.2 (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery)\n",
      "  Downloading grpcio-1.65.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)\n",
      "Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading hdfs-2.7.3.tar.gz (43 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting httplib2<0.23.0,>=0.8 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading httplib2-0.22.0-py3-none-any.whl.metadata (2.6 kB)\n",
      "Requirement already satisfied: jsonschema<5.0.0,>=4.0.0 in /opt/conda/lib/python3.10/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (4.25.1)\n",
      "Collecting jsonpickle<4.0.0,>=3.0.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading jsonpickle-3.4.2-py3-none-any.whl.metadata (8.1 kB)\n",
      "Collecting objsize<0.8.0,>=0.6.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading objsize-0.7.1-py3-none-any.whl.metadata (12 kB)\n",
      "Collecting pymongo<5.0.0,>=3.8.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading pymongo-4.15.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (22 kB)\n",
      "Collecting pydot<2,>=1.2.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading pydot-1.4.2-py2.py3-none-any.whl.metadata (8.0 kB)\n",
      "Collecting redis<6,>=5.0.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading redis-5.3.1-py3-none-any.whl.metadata (9.2 kB)\n",
      "Collecting regex>=2020.6.8 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading regex-2025.9.18-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (40 kB)\n",
      "Collecting sortedcontainers>=2.4.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl.metadata (10 kB)\n",
      "Requirement already satisfied: zstandard<1,>=0.18.0 in /opt/conda/lib/python3.10/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.25.0)\n",
      "Requirement already satisfied: pyyaml<7.0.0,>=3.12 in /opt/conda/lib/python3.10/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (6.0.3)\n",
      "Collecting beartype<0.22.0,>=0.21.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading beartype-0.21.0-py3-none-any.whl.metadata (33 kB)\n",
      "Collecting pyarrow-hotfix<1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading pyarrow_hotfix-0.7-py3-none-any.whl.metadata (3.6 kB)\n",
      "Collecting google-apitools<0.5.32,>=0.5.31 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google-apitools-0.5.31.tar.gz (173 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: google-auth-httplib2<0.3.0,>=0.1.0 in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.2.0)\n",
      "Requirement already satisfied: google-cloud-datastore<3,>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.21.0)\n",
      "Collecting google-cloud-pubsub<3,>=2.1.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_pubsub-2.31.1-py3-none-any.whl.metadata (10 kB)\n",
      "Collecting google-cloud-pubsublite<2,>=1.2.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_pubsublite-1.12.0-py2.py3-none-any.whl.metadata (5.8 kB)\n",
      "Requirement already satisfied: google-cloud-storage<3,>=2.18.2 in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.19.0)\n",
      "Requirement already satisfied: google-cloud-bigquery-storage<3,>=2.6.3 in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.33.1)\n",
      "Collecting google-cloud-bigtable<3,>=2.19.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_bigtable-2.33.0-py3-none-any.whl.metadata (5.7 kB)\n",
      "Collecting google-cloud-spanner<4,>=3.0.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_spanner-3.58.0-py3-none-any.whl.metadata (11 kB)\n",
      "Collecting google-cloud-dlp<4,>=3.0.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_dlp-3.32.0-py3-none-any.whl.metadata (9.7 kB)\n",
      "Requirement already satisfied: google-cloud-language<3,>=2.0 in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.17.2)\n",
      "Collecting google-cloud-videointelligence<3,>=2.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_videointelligence-2.16.2-py3-none-any.whl.metadata (10 kB)\n",
      "Collecting google-cloud-vision<4,>=2 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_vision-3.10.2-py3-none-any.whl.metadata (9.6 kB)\n",
      "Collecting google-cloud-recommendations-ai<0.11.0,>=0.1.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading google_cloud_recommendations_ai-0.10.18-py3-none-any.whl.metadata (9.7 kB)\n",
      "Collecting cloud-sql-python-connector<2.0.0,>=1.18.2 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading cloud_sql_python_connector-1.18.5-py3-none-any.whl.metadata (30 kB)\n",
      "Collecting python-tds>=1.16.1 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading python_tds-1.17.1-py3-none-any.whl.metadata (804 bytes)\n",
      "Collecting pg8000>=1.31.1 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading pg8000-1.31.5-py3-none-any.whl.metadata (88 kB)\n",
      "Collecting PyMySQL>=1.1.0 (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading pymysql-1.1.2-py3-none-any.whl.metadata (4.3 kB)\n",
      "Requirement already satisfied: keyrings.google-artifactregistry-auth in /opt/conda/lib/python3.10/site-packages (from apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.1.2)\n",
      "Requirement already satisfied: google-cloud-resource-manager<3.0.0,>=1.3.3 in /opt/conda/lib/python3.10/site-packages (from google-cloud-aiplatform) (1.14.2)\n",
      "Requirement already satisfied: shapely<3.0.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-aiplatform) (2.1.2)\n",
      "Requirement already satisfied: google-genai<2.0.0,>=1.37.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-aiplatform) (1.42.0)\n",
      "Requirement already satisfied: pydantic<3 in /opt/conda/lib/python3.10/site-packages (from google-cloud-aiplatform) (2.12.0)\n",
      "Requirement already satisfied: docstring_parser<1 in /opt/conda/lib/python3.10/site-packages (from google-cloud-aiplatform) (0.17.0)\n",
      "Requirement already satisfied: aiofiles in /opt/conda/lib/python3.10/site-packages (from cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (22.1.0)\n",
      "Requirement already satisfied: aiohttp in /opt/conda/lib/python3.10/site-packages (from cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (3.13.0)\n",
      "Requirement already satisfied: cryptography>=42.0.0 in /opt/conda/lib/python3.10/site-packages (from cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (46.0.2)\n",
      "Collecting dnspython>=2.0.0 (from cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading dnspython-2.8.0-py3-none-any.whl.metadata (5.7 kB)\n",
      "Collecting uritemplate<4dev,>=3.0.0 (from google-api-python-client<2,>=1.7.11->tfx-bsl)\n",
      "  Downloading uritemplate-3.0.1-py2.py3-none-any.whl.metadata (4.6 kB)\n",
      "Requirement already satisfied: oauth2client>=1.4.12 in /opt/conda/lib/python3.10/site-packages (from google-apitools<0.5.32,>=0.5.31->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (4.1.3)\n",
      "Requirement already satisfied: grpc-google-iam-v1<1.0.0,>=0.12.4 in /opt/conda/lib/python3.10/site-packages (from google-cloud-bigtable<3,>=2.19.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.14.2)\n",
      "Requirement already satisfied: opentelemetry-api>=1.27.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.37.0)\n",
      "Requirement already satisfied: opentelemetry-sdk>=1.27.0 in /opt/conda/lib/python3.10/site-packages (from google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.37.0)\n",
      "Collecting grpcio-status<2.0.0,>=1.33.2 (from google-api-core[grpc]<3.0.0,>=2.11.1->google-cloud-bigquery)\n",
      "  Downloading grpcio_status-1.76.0rc1-py3-none-any.whl.metadata (1.1 kB)\n",
      "Requirement already satisfied: overrides<8.0.0,>=6.0.1 in /opt/conda/lib/python3.10/site-packages (from google-cloud-pubsublite<2,>=1.2.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (7.7.0)\n",
      "Requirement already satisfied: sqlparse>=0.4.4 in /opt/conda/lib/python3.10/site-packages (from google-cloud-spanner<4,>=3.0.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.5.3)\n",
      "Collecting grpc-interceptor>=0.15.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading grpc_interceptor-0.15.4-py3-none-any.whl.metadata (8.4 kB)\n",
      "Requirement already satisfied: anyio<5.0.0,>=4.8.0 in /opt/conda/lib/python3.10/site-packages (from google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (4.11.0)\n",
      "Requirement already satisfied: httpx<1.0.0,>=0.28.1 in /opt/conda/lib/python3.10/site-packages (from google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (0.28.1)\n",
      "Requirement already satisfied: tenacity<9.2.0,>=8.2.3 in /opt/conda/lib/python3.10/site-packages (from google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (9.1.2)\n",
      "Requirement already satisfied: websockets<15.1.0,>=13.0.0 in /opt/conda/lib/python3.10/site-packages (from google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (15.0.1)\n",
      "Requirement already satisfied: exceptiongroup>=1.0.2 in /opt/conda/lib/python3.10/site-packages (from anyio<5.0.0,>=4.8.0->google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (1.3.0)\n",
      "Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.10/site-packages (from anyio<5.0.0,>=4.8.0->google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (1.3.1)\n",
      "Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading docopt-0.6.2.tar.gz (25 kB)\n",
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /opt/conda/lib/python3.10/site-packages (from httplib2<0.23.0,>=0.8->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (3.2.5)\n",
      "Requirement already satisfied: httpcore==1.* in /opt/conda/lib/python3.10/site-packages (from httpx<1.0.0,>=0.28.1->google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (1.0.9)\n",
      "Requirement already satisfied: h11>=0.16 in /opt/conda/lib/python3.10/site-packages (from httpcore==1.*->httpx<1.0.0,>=0.28.1->google-genai<2.0.0,>=1.37.0->google-cloud-aiplatform) (0.16.0)\n",
      "Requirement already satisfied: attrs>=22.2.0 in /opt/conda/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (25.4.0)\n",
      "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/conda/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2025.9.1)\n",
      "Requirement already satisfied: referencing>=0.28.4 in /opt/conda/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.36.2)\n",
      "Requirement already satisfied: rpds-py>=0.7.1 in /opt/conda/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.27.1)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in /opt/conda/lib/python3.10/site-packages (from pydantic<3->google-cloud-aiplatform) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.41.1 in /opt/conda/lib/python3.10/site-packages (from pydantic<3->google-cloud-aiplatform) (2.41.1)\n",
      "Requirement already satisfied: typing-inspection>=0.4.2 in /opt/conda/lib/python3.10/site-packages (from pydantic<3->google-cloud-aiplatform) (0.4.2)\n",
      "Collecting PyJWT>=2.9.0 (from redis<6,>=5.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading PyJWT-2.10.1-py3-none-any.whl.metadata (4.0 kB)\n",
      "Requirement already satisfied: async-timeout>=4.0.3 in /opt/conda/lib/python3.10/site-packages (from redis<6,>=5.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (5.0.1)\n",
      "Collecting astunparse>=1.6.0 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading astunparse-1.6.3-py2.py3-none-any.whl.metadata (4.4 kB)\n",
      "Collecting flatbuffers>=23.5.26 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading flatbuffers-25.9.23-py2.py3-none-any.whl.metadata (875 bytes)\n",
      "Collecting gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading gast-0.6.0-py3-none-any.whl.metadata (1.3 kB)\n",
      "Collecting google-pasta>=0.1.1 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading google_pasta-0.2.0-py3-none-any.whl.metadata (814 bytes)\n",
      "Collecting h5py>=3.10.0 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading h5py-3.15.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (3.0 kB)\n",
      "Collecting libclang>=13.0.0 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl.metadata (5.2 kB)\n",
      "Collecting ml-dtypes~=0.3.1 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading ml_dtypes-0.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)\n",
      "Collecting opt-einsum>=2.3.2 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading opt_einsum-3.4.0-py3-none-any.whl.metadata (6.3 kB)\n",
      "Collecting termcolor>=1.1.0 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading termcolor-3.1.0-py3-none-any.whl.metadata (6.4 kB)\n",
      "Requirement already satisfied: wrapt>=1.11.0 in /opt/conda/lib/python3.10/site-packages (from tensorflow<2.17,>=2.16->tensorflow-data-validation) (1.17.3)\n",
      "Collecting tensorboard<2.17,>=2.16 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading tensorboard-2.16.2-py3-none-any.whl.metadata (1.6 kB)\n",
      "Collecting keras>=3.0.0 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading keras-3.11.3-py3-none-any.whl.metadata (5.9 kB)\n",
      "Collecting tensorflow-io-gcs-filesystem>=0.23.1 (from tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading tensorflow_io_gcs_filesystem-0.37.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)\n",
      "Collecting markdown>=2.6.8 (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading markdown-3.9-py3-none-any.whl.metadata (5.1 kB)\n",
      "Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl.metadata (1.1 kB)\n",
      "Collecting werkzeug>=1.0.1 (from tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading werkzeug-3.1.3-py3-none-any.whl.metadata (3.7 kB)\n",
      "INFO: pip is looking at multiple versions of tensorflow-serving-api to determine which version is compatible with other requirements. This could take a while.\n",
      "Collecting tensorflow-serving-api<3,>=2.13.0 (from tfx-bsl)\n",
      "  Downloading tensorflow_serving_api-2.19.0-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "  Downloading tensorflow_serving_api-2.18.1-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "  Downloading tensorflow_serving_api-2.18.0-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "  Downloading tensorflow_serving_api-2.17.1-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "  Downloading tensorflow_serving_api-2.17.0-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "  Downloading tensorflow_serving_api-2.16.1-py2.py3-none-any.whl.metadata (1.8 kB)\n",
      "INFO: pip is looking at multiple versions of db-dtypes to determine which version is compatible with other requirements. This could take a while.\n",
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in /opt/conda/lib/python3.10/site-packages (from astunparse>=1.6.0->tensorflow<2.17,>=2.16->tensorflow-data-validation) (0.45.1)\n",
      "Requirement already satisfied: cffi>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from cryptography>=42.0.0->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.0.0)\n",
      "Requirement already satisfied: pycparser in /opt/conda/lib/python3.10/site-packages (from cffi>=2.0.0->cryptography>=42.0.0->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.22)\n",
      "Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.10/site-packages (from google-auth-oauthlib>=0.7.0->pandas-gbq) (2.0.0)\n",
      "Requirement already satisfied: rich in /opt/conda/lib/python3.10/site-packages (from keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation) (13.9.4)\n",
      "Collecting namex (from keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading namex-0.1.0-py3-none-any.whl.metadata (322 bytes)\n",
      "Collecting optree (from keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation)\n",
      "  Downloading optree-0.17.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (33 kB)\n",
      "Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in /opt/conda/lib/python3.10/site-packages (from opentelemetry-api>=1.27.0->google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (8.7.0)\n",
      "Requirement already satisfied: zipp>=3.20 in /opt/conda/lib/python3.10/site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api>=1.27.0->google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (3.23.0)\n",
      "Requirement already satisfied: opentelemetry-semantic-conventions==0.58b0 in /opt/conda/lib/python3.10/site-packages (from opentelemetry-sdk>=1.27.0->google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.58b0)\n",
      "Collecting scramp>=1.4.5 (from pg8000>=1.31.1->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading scramp-1.4.6-py3-none-any.whl.metadata (19 kB)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.10/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib>=0.7.0->pandas-gbq) (3.3.1)\n",
      "Collecting asn1crypto>=1.5.1 (from scramp>=1.4.5->pg8000>=1.31.1->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation)\n",
      "  Downloading asn1crypto-1.5.1-py2.py3-none-any.whl.metadata (13 kB)\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in /opt/conda/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard<2.17,>=2.16->tensorflow<2.17,>=2.16->tensorflow-data-validation) (3.0.3)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (2.6.1)\n",
      "Requirement already satisfied: aiosignal>=1.4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.4.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.8.0)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (6.7.0)\n",
      "Requirement already satisfied: propcache>=0.2.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.4.1)\n",
      "Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->cloud-sql-python-connector<2.0.0,>=1.18.2->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.22.0)\n",
      "Requirement already satisfied: keyring in /opt/conda/lib/python3.10/site-packages (from keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (25.6.0)\n",
      "Requirement already satisfied: pluggy in /opt/conda/lib/python3.10/site-packages (from keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.6.0)\n",
      "Requirement already satisfied: SecretStorage>=3.2 in /opt/conda/lib/python3.10/site-packages (from keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (3.4.0)\n",
      "Requirement already satisfied: jeepney>=0.4.2 in /opt/conda/lib/python3.10/site-packages (from keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (0.9.0)\n",
      "Requirement already satisfied: jaraco.classes in /opt/conda/lib/python3.10/site-packages (from keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (3.4.0)\n",
      "Requirement already satisfied: jaraco.functools in /opt/conda/lib/python3.10/site-packages (from keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (4.3.0)\n",
      "Requirement already satisfied: jaraco.context in /opt/conda/lib/python3.10/site-packages (from keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (6.0.1)\n",
      "Requirement already satisfied: more-itertools in /opt/conda/lib/python3.10/site-packages (from jaraco.classes->keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (10.8.0)\n",
      "Requirement already satisfied: backports.tarfile in /opt/conda/lib/python3.10/site-packages (from jaraco.context->keyring->keyrings.google-artifactregistry-auth->apache-beam[gcp]<3,>=2.47->tensorflow-data-validation) (1.2.0)\n",
      "Requirement already satisfied: markdown-it-py>=2.2.0 in /opt/conda/lib/python3.10/site-packages (from rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation) (4.0.0)\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/lib/python3.10/site-packages (from rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation) (2.19.2)\n",
      "Requirement already satisfied: mdurl~=0.1 in /opt/conda/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.0.0->tensorflow<2.17,>=2.16->tensorflow-data-validation) (0.1.2)\n",
      "Using cached pandas-1.5.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)\n",
      "Downloading grpcio_status-1.48.2-py3-none-any.whl (14 kB)\n",
      "Downloading tensorflow_data_validation-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.0 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.0/19.0 MB\u001b[0m \u001b[31m88.8 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m6m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tensorflow_metadata-1.16.1-py3-none-any.whl (28 kB)\n",
      "Downloading protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m46.5 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading pyarrow-10.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (35.9 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m35.9/35.9 MB\u001b[0m \u001b[31m100.4 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tfx_bsl-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.5 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m22.5/22.5 MB\u001b[0m \u001b[31m119.8 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading absl_py-1.4.0-py3-none-any.whl (126 kB)\n",
      "Using cached apache_beam-2.68.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.6 MB)\n",
      "Using cached beartype-0.21.0-py3-none-any.whl (1.2 MB)\n",
      "Using cached cloud_sql_python_connector-1.18.5-py3-none-any.whl (49 kB)\n",
      "Downloading fastavro-1.12.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (3.2 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.2/3.2 MB\u001b[0m \u001b[31m97.5 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading fasteners-0.20-py3-none-any.whl (18 kB)\n",
      "Using cached google_api_python_client-1.12.11-py2.py3-none-any.whl (62 kB)\n",
      "Using cached google_cloud_bigtable-2.33.0-py3-none-any.whl (537 kB)\n",
      "Downloading google_cloud_dlp-3.32.0-py3-none-any.whl (216 kB)\n",
      "Downloading google_cloud_pubsub-2.31.1-py3-none-any.whl (319 kB)\n",
      "Downloading google_cloud_pubsublite-1.12.0-py2.py3-none-any.whl (322 kB)\n",
      "Downloading google_cloud_recommendations_ai-0.10.18-py3-none-any.whl (212 kB)\n",
      "Downloading google_cloud_spanner-3.58.0-py3-none-any.whl (501 kB)\n",
      "Using cached google_cloud_videointelligence-2.16.2-py3-none-any.whl (275 kB)\n",
      "Downloading google_cloud_vision-3.10.2-py3-none-any.whl (527 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m527.9/527.9 kB\u001b[0m \u001b[31m21.8 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hUsing cached grpcio-1.65.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.7 MB)\n",
      "Downloading httplib2-0.22.0-py3-none-any.whl (96 kB)\n",
      "Using cached jsonpickle-3.4.2-py3-none-any.whl (46 kB)\n",
      "Downloading objsize-0.7.1-py3-none-any.whl (11 kB)\n",
      "Downloading orjson-3.11.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (132 kB)\n",
      "Using cached pyarrow_hotfix-0.7-py3-none-any.whl (7.9 kB)\n",
      "Downloading pydot-1.4.2-py2.py3-none-any.whl (21 kB)\n",
      "Downloading pymongo-4.15.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (1.2 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m61.9 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hUsing cached dnspython-2.8.0-py3-none-any.whl (331 kB)\n",
      "Using cached redis-5.3.1-py3-none-any.whl (272 kB)\n",
      "Downloading tensorflow-2.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (590.6 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m590.6/590.6 MB\u001b[0m \u001b[31m52.3 MB/s\u001b[0m  \u001b[33m0:00:06\u001b[0mm0:00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hDownloading ml_dtypes-0.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m98.0 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading tensorboard-2.16.2-py3-none-any.whl (5.5 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.5/5.5 MB\u001b[0m \u001b[31m120.8 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hUsing cached tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)\n",
      "Downloading tensorflow_serving_api-2.16.1-py2.py3-none-any.whl (26 kB)\n",
      "Using cached uritemplate-3.0.1-py2.py3-none-any.whl (15 kB)\n",
      "Downloading db_dtypes-1.4.2-py2.py3-none-any.whl (18 kB)\n",
      "Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)\n",
      "Downloading flatbuffers-25.9.23-py2.py3-none-any.whl (30 kB)\n",
      "Downloading gast-0.6.0-py3-none-any.whl (21 kB)\n",
      "Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)\n",
      "Downloading grpc_interceptor-0.15.4-py3-none-any.whl (20 kB)\n",
      "Downloading h5py-3.15.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (4.7 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.7/4.7 MB\u001b[0m \u001b[31m126.2 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hUsing cached keras-3.11.3-py3-none-any.whl (1.4 MB)\n",
      "Downloading libclang-18.1.1-py2.py3-none-manylinux2010_x86_64.whl (24.5 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m24.5/24.5 MB\u001b[0m \u001b[31m129.9 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading markdown-3.9-py3-none-any.whl (107 kB)\n",
      "Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)\n",
      "Using cached pg8000-1.31.5-py3-none-any.whl (57 kB)\n",
      "Using cached PyJWT-2.10.1-py3-none-any.whl (22 kB)\n",
      "Using cached pymysql-1.1.2-py3-none-any.whl (45 kB)\n",
      "Using cached python_tds-1.17.1-py3-none-any.whl (86 kB)\n",
      "Downloading regex-2025.9.18-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (789 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m789.9/789.9 kB\u001b[0m \u001b[31m40.5 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hUsing cached scramp-1.4.6-py3-none-any.whl (12 kB)\n",
      "Using cached asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)\n",
      "Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)\n",
      "Downloading tensorflow_io_gcs_filesystem-0.37.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.1 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.1/5.1 MB\u001b[0m \u001b[31m113.0 MB/s\u001b[0m  \u001b[33m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading termcolor-3.1.0-py3-none-any.whl (7.7 kB)\n",
      "Downloading werkzeug-3.1.3-py3-none-any.whl (224 kB)\n",
      "Using cached namex-0.1.0-py3-none-any.whl (5.9 kB)\n",
      "Using cached optree-0.17.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (387 kB)\n",
      "Building wheels for collected packages: crcmod, dill, google-apitools, hdfs, docopt\n",
      "\u001b[33m  DEPRECATION: Building 'crcmod' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'crcmod'. Discussion can be found at https://github.com/pypa/pip/issues/6334\u001b[0m\u001b[33m\n",
      "\u001b[0m  Building wheel for crcmod (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for crcmod: filename=crcmod-1.7-cp310-cp310-linux_x86_64.whl size=23168 sha256=a281c67d815ddafb8bca51ff38048c977efdb7d13bd14656b4b452553ed56654\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/85/4c/07/72215c529bd59d67e3dac29711d7aba1b692f543c808ba9e86\n",
      "\u001b[33m  DEPRECATION: Building 'dill' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'dill'. Discussion can be found at https://github.com/pypa/pip/issues/6334\u001b[0m\u001b[33m\n",
      "\u001b[0m  Building wheel for dill (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for dill: filename=dill-0.3.1.1-py3-none-any.whl size=78635 sha256=742843fb4987a01929f040438db19588969a1fee2780bcc6d4b187fcc21e2e3c\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/ea/e2/86/64980d90e297e7bf2ce588c2b96e818f5399c515c4bb8a7e4f\n",
      "\u001b[33m  DEPRECATION: Building 'google-apitools' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'google-apitools'. Discussion can be found at https://github.com/pypa/pip/issues/6334\u001b[0m\u001b[33m\n",
      "\u001b[0m  Building wheel for google-apitools (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for google-apitools: filename=google_apitools-0.5.31-py3-none-any.whl size=131077 sha256=babe825b538e9e21cdf9d9b2a422c42802871af01f96509ee886c49b97ea269c\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/04/b7/e0/9712f8c23a5da3d9d16fb88216b897bf60e85b12f5470f26ee\n",
      "\u001b[33m  DEPRECATION: Building 'hdfs' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'hdfs'. Discussion can be found at https://github.com/pypa/pip/issues/6334\u001b[0m\u001b[33m\n",
      "\u001b[0m  Building wheel for hdfs (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for hdfs: filename=hdfs-2.7.3-py3-none-any.whl size=34431 sha256=dccca0cb386584763749a389f77799c6203f0f37cc357cd0e7bd6535fd112133\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/e5/8d/b6/99c1c0a3ac5788c866b0ecd3f48b0134a5910e6ed26011800b\n",
      "\u001b[33m  DEPRECATION: Building 'docopt' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'docopt'. Discussion can be found at https://github.com/pypa/pip/issues/6334\u001b[0m\u001b[33m\n",
      "\u001b[0m  Building wheel for docopt (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13783 sha256=c1b26c3053f7202643701cfe28e037955210a977b635c6135f6042417e701c07\n",
      "  Stored in directory: /home/jupyter/.cache/pip/wheels/fc/ab/d4/5da2067ac95b36618c629a5f93f809425700506f72c9732fac\n",
      "Successfully built crcmod dill google-apitools hdfs docopt\n",
      "Installing collected packages: sortedcontainers, python-tds, pyfarmhash, namex, libclang, flatbuffers, docopt, crcmod, asn1crypto, werkzeug, uritemplate, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, scramp, regex, PyMySQL, PyJWT, pydot, pyarrow-hotfix, pyarrow, protobuf, orjson, optree, opt-einsum, objsize, ml-dtypes, markdown, jsonpickle, httplib2, h5py, grpcio, google-pasta, gast, fasteners, fastavro, dnspython, dill, beartype, astunparse, absl-py, tensorflow-metadata, tensorboard, redis, pymongo, pg8000, pandas, hdfs, grpc-interceptor, keras, grpcio-status, google-apitools, db-dtypes, tensorflow, google-api-python-client, cloud-sql-python-connector, apache-beam, tensorflow-serving-api, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-dlp, google-cloud-bigtable, google-cloud-pubsublite, tfx-bsl, tensorflow-data-validation\n",
      "\u001b[2K  Attempting uninstall: uritemplate━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m 9/68\u001b[0m [werkzeug]\n",
      "\u001b[2K    Found existing installation: uritemplate 4.2.0━━━━━━━━━━━━\u001b[0m \u001b[32m 9/68\u001b[0m [werkzeug]\n",
      "\u001b[2K    Uninstalling uritemplate-4.2.0:━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m 9/68\u001b[0m [werkzeug]\n",
      "\u001b[2K      Successfully uninstalled uritemplate-4.2.0━━━━━━━━━━━━━━\u001b[0m \u001b[32m 9/68\u001b[0m [werkzeug]\n",
      "\u001b[2K  Attempting uninstall: pyarrow\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m16/68\u001b[0m [PyMySQL]ard-data-server]stem]\n",
      "\u001b[2K    Found existing installation: pyarrow 21.0.0━━━━━━━━━━━━━━━\u001b[0m \u001b[32m16/68\u001b[0m [PyMySQL]\n",
      "\u001b[2K    Uninstalling pyarrow-21.0.0:━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m16/68\u001b[0m [PyMySQL]\n",
      "\u001b[2K      Successfully uninstalled pyarrow-21.0.0━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m16/68\u001b[0m [PyMySQL]\n",
      "\u001b[2K  Attempting uninstall: protobufm\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20/68\u001b[0m [pyarrow]\n",
      "\u001b[2K    Found existing installation: protobuf 6.31.1━━━━━━━━━━━━━━\u001b[0m \u001b[32m20/68\u001b[0m [pyarrow]\n",
      "\u001b[2K    Uninstalling protobuf-6.31.1:m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20/68\u001b[0m [pyarrow]\n",
      "\u001b[2K      Successfully uninstalled protobuf-6.31.1━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20/68\u001b[0m [pyarrow]\n",
      "\u001b[2K   \u001b[91m━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21/68\u001b[0m [protobuf]\u001b[33m  WARNING: Failed to remove contents in a temporary directory '/opt/conda/lib/python3.10/site-packages/google/~rotobuf'.\n",
      "  You can safely remove it manually.\u001b[0m\u001b[33m\n",
      "\u001b[2K  Attempting uninstall: httplib2╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m26/68\u001b[0m [ml-dtypes]]\n",
      "\u001b[2K    Found existing installation: httplib2 0.31.0━━━━━━━━━━━━━━\u001b[0m \u001b[32m26/68\u001b[0m [ml-dtypes]\n",
      "\u001b[2K    Uninstalling httplib2-0.31.0:\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m26/68\u001b[0m [ml-dtypes]\n",
      "\u001b[2K      Successfully uninstalled httplib2-0.31.0━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m26/68\u001b[0m [ml-dtypes]\n",
      "\u001b[2K  Attempting uninstall: grpcio[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m30/68\u001b[0m [h5py]ib2]\n",
      "\u001b[2K    Found existing installation: grpcio 1.75.1━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m30/68\u001b[0m [h5py]\n",
      "\u001b[2K    Uninstalling grpcio-1.75.1:[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m31/68\u001b[0m [grpcio]\n",
      "\u001b[2K      Successfully uninstalled grpcio-1.75.1━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m31/68\u001b[0m [grpcio]\n",
      "\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m31/68\u001b[0m [grpcio]\u001b[33m  WARNING: Failed to remove contents in a temporary directory '/opt/conda/lib/python3.10/site-packages/~rpc'.\n",
      "  You can safely remove it manually.\u001b[0m\u001b[33m\n",
      "\u001b[2K  Attempting uninstall: absl-py[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38/68\u001b[0m [beartype]]\n",
      "\u001b[2K    Found existing installation: absl-py 2.3.1━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38/68\u001b[0m [beartype]\n",
      "\u001b[2K    Uninstalling absl-py-2.3.1:[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38/68\u001b[0m [beartype]\n",
      "\u001b[2K      Successfully uninstalled absl-py-2.3.1m\u001b[90m━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40/68\u001b[0m [absl-py]\n",
      "\u001b[2K  Attempting uninstall: pandas━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━\u001b[0m \u001b[32m44/68\u001b[0m [pymongo]ard]\n",
      "\u001b[2K    Found existing installation: pandas 2.3.390m━━━━━━━━━━━━━━\u001b[0m \u001b[32m44/68\u001b[0m [pymongo]\n",
      "\u001b[2K    Uninstalling pandas-2.3.3:━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━\u001b[0m \u001b[32m46/68\u001b[0m [pandas]\n",
      "\u001b[2K      Successfully uninstalled pandas-2.3.30m\u001b[90m━━━━━━━━━━━━\u001b[0m \u001b[32m46/68\u001b[0m [pandas]\n",
      "\u001b[2K  Attempting uninstall: grpcio-status[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━\u001b[0m \u001b[32m49/68\u001b[0m [keras]]\n",
      "\u001b[2K    Found existing installation: grpcio-status 1.75.1━━━━━━━━━\u001b[0m \u001b[32m49/68\u001b[0m [keras]\n",
      "\u001b[2K    Uninstalling grpcio-status-1.75.1:91m╸\u001b[0m\u001b[90m━━━━━━━━━━━\u001b[0m \u001b[32m49/68\u001b[0m [keras]\n",
      "\u001b[2K      Successfully uninstalled grpcio-status-1.75.1━━━━━━━━━━━\u001b[0m \u001b[32m49/68\u001b[0m [keras]\n",
      "\u001b[2K   \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━\u001b[0m \u001b[32m50/68\u001b[0m [grpcio-status]\u001b[33m  WARNING: Failed to remove contents in a temporary directory '/opt/conda/lib/python3.10/site-packages/~rpc_status'.\n",
      "  You can safely remove it manually.\u001b[0m\u001b[33m\n",
      "\u001b[2K  Attempting uninstall: db-dtypes━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━\u001b[0m \u001b[32m51/68\u001b[0m [google-apitools]\n",
      "\u001b[2K    Found existing installation: db-dtypes 1.4.3\u001b[90m━━━━━━━━━\u001b[0m \u001b[32m51/68\u001b[0m [google-apitools]\n",
      "\u001b[2K    Uninstalling db-dtypes-1.4.3:━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━\u001b[0m \u001b[32m51/68\u001b[0m [google-apitools]\n",
      "\u001b[2K      Successfully uninstalled db-dtypes-1.4.30m\u001b[90m━━━━━━━━━\u001b[0m \u001b[32m51/68\u001b[0m [google-apitools]\n",
      "\u001b[2K  Attempting uninstall: google-api-python-client╺\u001b[0m\u001b[90m━━━━━━━━\u001b[0m \u001b[32m53/68\u001b[0m [tensorflow]]\n",
      "\u001b[2K    Found existing installation: google-api-python-client 2.184.0m \u001b[32m53/68\u001b[0m [tensorflow]\n",
      "\u001b[2K    Uninstalling google-api-python-client-2.184.0:[90m━━━━━━━━\u001b[0m \u001b[32m53/68\u001b[0m [tensorflow]\n",
      "\u001b[2K      Successfully uninstalled google-api-python-client-2.184.0[0m \u001b[32m53/68\u001b[0m [tensorflow]\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m68/68\u001b[0m [tensorflow-data-validation]lidation]blite]s-ai]\n",
      "\u001b[1A\u001b[2K\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "bigframes 2.24.0 requires pyarrow>=15.0.2, but you have pyarrow 10.0.1 which is incompatible.\n",
      "geopandas 1.1.1 requires pandas>=2.0.0, but you have pandas 1.5.3 which is incompatible.\n",
      "kfp 2.14.4 requires protobuf<7.0,==6.31.1, but you have protobuf 3.20.3 which is incompatible.\n",
      "kfp-pipeline-spec 2.14.0 requires protobuf<7.0,==6.31.1, but you have protobuf 3.20.3 which is incompatible.\n",
      "opentelemetry-proto 1.37.0 requires protobuf<7.0,>=5.0, but you have protobuf 3.20.3 which is incompatible.\n",
      "visions 0.8.1 requires pandas>=2.0.0, but you have pandas 1.5.3 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mSuccessfully installed PyJWT-2.10.1 PyMySQL-1.1.2 absl-py-1.4.0 apache-beam-2.68.0 asn1crypto-1.5.1 astunparse-1.6.3 beartype-0.21.0 cloud-sql-python-connector-1.18.5 crcmod-1.7 db-dtypes-1.4.2 dill-0.3.1.1 dnspython-2.8.0 docopt-0.6.2 fastavro-1.12.1 fasteners-0.20 flatbuffers-25.9.23 gast-0.6.0 google-api-python-client-1.12.11 google-apitools-0.5.31 google-cloud-bigtable-2.33.0 google-cloud-dlp-3.32.0 google-cloud-pubsub-2.31.1 google-cloud-pubsublite-1.12.0 google-cloud-recommendations-ai-0.10.18 google-cloud-spanner-3.58.0 google-cloud-videointelligence-2.16.2 google-cloud-vision-3.10.2 google-pasta-0.2.0 grpc-interceptor-0.15.4 grpcio-1.65.5 grpcio-status-1.48.2 h5py-3.15.0 hdfs-2.7.3 httplib2-0.22.0 jsonpickle-3.4.2 keras-3.11.3 libclang-18.1.1 markdown-3.9 ml-dtypes-0.3.2 namex-0.1.0 objsize-0.7.1 opt-einsum-3.4.0 optree-0.17.0 orjson-3.11.3 pandas-1.5.3 pg8000-1.31.5 protobuf-3.20.3 pyarrow-10.0.1 pyarrow-hotfix-0.7 pydot-1.4.2 pyfarmhash-0.3.2 pymongo-4.15.3 python-tds-1.17.1 redis-5.3.1 regex-2025.9.18 scramp-1.4.6 sortedcontainers-2.4.0 tensorboard-2.16.2 tensorboard-data-server-0.7.2 tensorflow-2.16.2 tensorflow-data-validation-1.16.1 tensorflow-io-gcs-filesystem-0.37.1 tensorflow-metadata-1.16.1 tensorflow-serving-api-2.16.1 termcolor-3.1.0 tfx-bsl-1.16.1 uritemplate-3.0.1 werkzeug-3.1.3\n"
     ]
    }
   ],
   "source": [
    "!pip install \\\n",
    "    google-cloud-bigquery \\\n",
    "    pandas==1.5.3 \\\n",
    "    pandas-gbq \\\n",
    "    pyarrow \\\n",
    "    tensorflow-data-validation \\\n",
    "    google-cloud-aiplatform \\\n",
    "    tensorflow-metadata \\\n",
    "    google-api-core \\\n",
    "    google-auth \\\n",
    "    google-cloud-core \\\n",
    "    google-resumable-media \\\n",
    "    tfx-bsl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "02nrfqgSOsw2"
   },
   "source": [
    "Check that the version of google-cloud-aiplatform is 1.51.0 or later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "uq2kSzjTOuQp",
    "outputId": "5b04eb95-43c7-4b82-97b4-1bab15e54a92",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.120.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from google.cloud import aiplatform\n",
    "\n",
    "aiplatform.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U4o9GjHNfHdI"
   },
   "source": [
    "### Restart runtime (Colab only)\n",
    "\n",
    "To use the newly installed packages, you must restart the runtime on Google Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "JaY8Q_T3fA38",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "if not \"google.colab\" in sys.modules:\n",
    "\n",
    "    import IPython\n",
    "\n",
    "    app = IPython.Application.instance()\n",
    "    app.kernel.do_shutdown(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4LvVXIhRO0Vk"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>⚠️ The kernel is going to restart. Please wait until it is finished before continuing to the next step. ⚠️</b>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i5-StiZvO3-6"
   },
   "source": [
    "### Authenticate your notebook environment (Colab only)\n",
    "\n",
    "Authenticate your environment on Google Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "YIXcOK0uO5q_",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "if \"google.colab\" in sys.modules:\n",
    "\n",
    "    from google.colab import auth\n",
    "\n",
    "    auth.authenticate_user()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uHVx1c-OO7KR"
   },
   "source": [
    "### Set Google Cloud project information and initialize Vertex AI SDK\n",
    "\n",
    "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gg6xkyR7O9cU",
    "outputId": "628f58ab-9605-4328-e437-4024e23a3371",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated property [core/project].\n",
      "Updated property [ai/region].\n"
     ]
    }
   ],
   "source": [
    "# Make sure to replace the value of `PROJECT_ID` with your GCP project ID and `LOCATION` with the region associated with your lab.\n",
    "PROJECT_ID = \"qwiklabs-gcp-01-89e8598a3363\"  # @param {type:\"string\"}\n",
    "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
    "\n",
    "\n",
    "import os\n",
    "\n",
    "import vertexai\n",
    "\n",
    "! gcloud config set project $PROJECT_ID\n",
    "os.environ[\"GOOGLE_CLOUD_PROJECT\"] = PROJECT_ID\n",
    "! gcloud config set ai/region $LOCATION\n",
    "\n",
    "vertexai.init(project=PROJECT_ID, location=LOCATION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LMRdCWwqPBpR"
   },
   "source": [
    "## Start Model Monitoring tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l_BdNCfk7Hjz"
   },
   "source": [
    "### Step 1: Create a Cloud Storage bucket\n",
    "\n",
    "Create a Cloud Storage bucket to store intermediate artifacts such as datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "3_CZRJ5L60Mi",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a Cloud Storage bucket\n",
    "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lvHsyHsi7Dg8"
   },
   "source": [
    "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mrHlJi-e7AyX",
    "outputId": "db42623b-c351-4442-f4f5-e0db06719b44",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/...\n"
     ]
    }
   ],
   "source": [
    "! gsutil mb -l {LOCATION} -p {PROJECT_ID} {BUCKET_URI}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5ZLxDjB6lYXx"
   },
   "source": [
    "### Step 2: Prepare a model in Vertex AI Model Registry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1_OchKcp60FE"
   },
   "source": [
    "You can register a model in Vertex AI Model Registry with its artifacts, enabling you to perform online serving or batch prediction. Alternatively, you can register a referenced/placeholder model that includes only the model's name.\n",
    "In this notebook, you register a model with artifacts because you'll run a batch prediction job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SwrD1CnmmJsk",
    "outputId": "a309aea8-03d8-43b3-ac6b-7dddc876a3e0",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import google.cloud.aiplatform as aiplatform\n",
    "\n",
    "MODEL_PATH = \"gs://cloud-samples-data/vertex-ai/model-deployment/models/churn\"\n",
    "MODEL_NAME = \"churn\"\n",
    "IMAGE = \"us-docker.pkg.dev/cloud-aiplatform/prediction/tf2-cpu.2-5:latest\"\n",
    "\n",
    "model = aiplatform.Model.upload(\n",
    "    display_name=MODEL_NAME,\n",
    "    artifact_uri=MODEL_PATH,\n",
    "    serving_container_image_uri=IMAGE,\n",
    "    sync=True,\n",
    ")\n",
    "\n",
    "MODEL_ID = model.resource_name.split(\"/\")[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RuwAoHTilv1j"
   },
   "source": [
    "### Step 3: Create a model monitor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Wp6jVFGvC6wt"
   },
   "source": [
    "Create a model monitor to associate monitoring details with a model version that has been register in Vertex AI Model Registry."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9hH4wSCXfgoE"
   },
   "source": [
    "#### Define Model Monitoring Schema"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jyg0cMAdfrAa"
   },
   "source": [
    "The monitoring schema is required for model monitors. It includes the names of input features, prediction outputs and, if available, ground truths, along with their respective data type.\n",
    "\n",
    "**Note: For AutoML tables (regression and classification), defining the schema is optional. The schema is automatically fetched when available. If Vertex AI cannot get the schema information, you must provide it.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "GR-4eTn3fkHk",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2025-10-15 14:19:57.426995: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.\n",
      "2025-10-15 14:19:57.432662: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.\n",
      "2025-10-15 14:19:57.451885: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:479] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2025-10-15 14:19:57.487349: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:10575] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2025-10-15 14:19:57.487411: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1442] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "2025-10-15 14:19:57.515133: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    }
   ],
   "source": [
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "MODEL_MONITORING_SCHEMA = ml_monitoring.spec.ModelMonitoringSchema(\n",
    "    feature_fields=[\n",
    "        ml_monitoring.spec.FieldSchema(name=\"user_pseudo_id\", data_type=\"string\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"country\", data_type=\"string\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"operating_system\", data_type=\"string\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"cnt_user_engagement\", data_type=\"integer\"),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_level_start_quickplay\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_level_end_quickplay\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_level_complete_quickplay\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_level_reset_quickplay\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"cnt_post_score\", data_type=\"integer\"),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_spend_virtual_currency\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"cnt_ad_reward\", data_type=\"integer\"),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_challenge_a_friend\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"cnt_completed_5_levels\", data_type=\"integer\"\n",
    "        ),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"cnt_use_extra_steps\", data_type=\"integer\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"month\", data_type=\"categorical\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"julianday\", data_type=\"integer\"),\n",
    "        ml_monitoring.spec.FieldSchema(name=\"dayofweek\", data_type=\"integer\"),\n",
    "    ],\n",
    "    ground_truth_fields=[\n",
    "        ml_monitoring.spec.FieldSchema(name=\"churned\", data_type=\"categorical\")\n",
    "    ],\n",
    "    prediction_fields=[\n",
    "        ml_monitoring.spec.FieldSchema(\n",
    "            name=\"predicted_churned\", data_type=\"categorical\"\n",
    "        )\n",
    "    ],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L8KuANNbiiEV"
   },
   "source": [
    "#### Define training dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SvFZRodti3qN"
   },
   "source": [
    "The training dataset can serve as the baseline dataset to calculate monitoring metrics. You can register the training dataset in the model monitor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CSqE4_roiw-q",
    "outputId": "44b555c1-41b4-41fe-bd05-4aa73f24a861",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_training.csv [Content-Type=text/csv]...\n",
      "/ [1 files][717.9 KiB/717.9 KiB]                                                \n",
      "Operation completed over 1 objects/717.9 KiB.                                    \n"
     ]
    }
   ],
   "source": [
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "# Copy files to your projects gs bucket to avoid permission issues.\n",
    "# Ignore any error(s) for bucket already exists.\n",
    "PUBLIC_TRAINING_DATASET = (\n",
    "    \"gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_training.csv\"\n",
    ")\n",
    "TRAINING_URI = f\"{BUCKET_URI}/model-monitoring/churn/churn_training.csv\"\n",
    "\n",
    "! gsutil copy $PUBLIC_TRAINING_DATASET $TRAINING_URI\n",
    "\n",
    "TRAINING_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    gcs_uri=TRAINING_URI, data_format=\"csv\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5ZnBZUEMjtDD"
   },
   "source": [
    "#### Create a model monitor resource"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OPBdgqz6jv9_"
   },
   "source": [
    "A model monitor is a top-level resource to manage your metrics and model monitoring jobs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "l_d917Ory595",
    "outputId": "777de5e0-fd25-47a9-b254-9aa75c3f3efe",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Creating ModelMonitor\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Create ModelMonitor backing LRO: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/operations/4985967833880985600\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MODEL MONITOR 2411541261015056384 created.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitor created. Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:To use this ModelMonitor in another session:\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:model_monitor = aiplatform.ModelMonitor('projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384')\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:https://console.cloud.google.com/vertex-ai/model-monitoring/locations/us-central1/model-monitors/2411541261015056384?project=qwiklabs-gcp-01-89e8598a3363\n"
     ]
    }
   ],
   "source": [
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "my_model_monitor = ml_monitoring.ModelMonitor.create(\n",
    "    project=PROJECT_ID,\n",
    "    location=LOCATION,\n",
    "    display_name=\"churn_model_monitor\",\n",
    "    model_name=model.resource_name,\n",
    "    model_version_id=\"1\",\n",
    "    training_dataset=TRAINING_DATASET,\n",
    "    model_monitoring_schema=MODEL_MONITORING_SCHEMA,\n",
    ")\n",
    "MODEL_MONITOR_ID = my_model_monitor.name\n",
    "print(f\"MODEL MONITOR {MODEL_MONITOR_ID} created.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dkHlwQuVli_2"
   },
   "source": [
    "### Step 4: Run an on-demand model monitoring job"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GpVp3fh3rkov"
   },
   "source": [
    "#### Define the monitoring objective configs\n",
    "\n",
    "For tabular models, Model Monitoring supports the following objectives:\n",
    "\n",
    "*   **Input feature drift detection**\n",
    "\n",
    "    Model Monitoring offers drift analysis for both categorical and numeric feature types, with the following supported metrics:\n",
    "\n",
    "    *    Categorical Feature: `Jensen Shannon Divergence`, `L Infinity`\n",
    "    *    Numeric Feature: `Jensen Shannon Divergence`\n",
    "\n",
    "    You can choose to analyze only the features of interest by specifying them in the `features` fields of the `ml_monitoring.spec.DataDriftSpec` specification. If not specified, all input features in the model schema are analyzed. Additionally, you have the option to set default thresholds for categorical or numeric features, or you can specify thresholds for individual features. If the detected drift surpasses a threshold, an alert is sent through email or another notification channel.\n",
    "\n",
    "*  **Prediction output drift detection**\n",
    "\n",
    "    Similar to input feature drift detection, prediction output drift detection identifies data drift in the prediction outputs.\n",
    "\n",
    "*   **Feature attribution drift detection**\n",
    "\n",
    "    Model Monitoring leverages Vertex Explainable AI to monitor feature attributions. Explainable AI enables you to understand the relative contribution of each feature to a resulting prediction. In essence, it assesses the magnitude of each feature's influence.\n",
    "    You must configure the `Explanation` specification with the feature attribution objectives configuration.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n49LgtHBfpTE"
   },
   "source": [
    "Input feature drift specification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "CtrsEblNqzLT",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "FEATURE_THRESHOLDS = {\n",
    "    \"country\": 0.003,\n",
    "    \"cnt_user_engagement\": 0.004,\n",
    "}\n",
    "\n",
    "FEATURE_DRIFT_SPEC = ml_monitoring.spec.DataDriftSpec(\n",
    "    categorical_metric_type=\"l_infinity\",\n",
    "    numeric_metric_type=\"jensen_shannon_divergence\",\n",
    "    default_categorical_alert_threshold=0.2,\n",
    "    default_numeric_alert_threshold=0.3,\n",
    "    feature_alert_thresholds=FEATURE_THRESHOLDS,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DsuNTFAeftgj"
   },
   "source": [
    "Prediction output drift specification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "tcUz6OFvfoBv",
    "tags": []
   },
   "outputs": [],
   "source": [
    "PREDICTION_OUTPUT_DRIFT_SPEC = ml_monitoring.spec.DataDriftSpec(\n",
    "    categorical_metric_type=\"l_infinity\",\n",
    "    numeric_metric_type=\"jensen_shannon_divergence\",\n",
    "    default_categorical_alert_threshold=0.1,\n",
    "    default_numeric_alert_threshold=0.1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ak54Y713f1aj"
   },
   "source": [
    "Feature attribution specification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "GgIZLRJa5QY4",
    "tags": []
   },
   "outputs": [],
   "source": [
    "FEATURE_ATTRIBUTION_SPEC = ml_monitoring.spec.FeatureAttributionSpec(\n",
    "    default_alert_threshold=0.0003,\n",
    "    feature_alert_thresholds={\"cnt_ad_reward\": 0.0001},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "avV5qcCUr9IT"
   },
   "source": [
    "#### Define the alert notification and metrics output spec."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8N9YlXzOvyOs"
   },
   "source": [
    "Model Monitoring supports the following notification methods:\n",
    "\n",
    "*   Email\n",
    "*   [Notification Channel](https://cloud.google.com/monitoring/support/notification-options)\n",
    "*   [Cloud Logging](https://cloud.google.com/logging/docs)  \n",
    "\n",
    "This notebook uses email as an example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7jzY1mpdw-CY"
   },
   "source": [
    "Export generated metrics to the Google Cloud Storage location that you specified or, if you don't specify a location, Vertex AI creates a default bucket to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "yclsJhNrsI-F",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "EMAIL = \"stripling@google.com\"  # @param {type:\"string\"}\n",
    "if os.getenv(\"IS_TESTING\"):\n",
    "    EMAIL = \"noreply@google.com\"\n",
    "\n",
    "NOTIFICATION_SPEC = ml_monitoring.spec.NotificationSpec(\n",
    "    user_emails=[EMAIL],\n",
    ")\n",
    "\n",
    "OUTPUT_SPEC = ml_monitoring.spec.OutputSpec(gcs_base_dir=BUCKET_URI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rQzbfaAFq9mV"
   },
   "source": [
    "#### Run Model Monitoring Jobs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "saA2MVZ-Yhy_"
   },
   "source": [
    "##### **Example 1: Detect feature drift by comparing a batch prediction job with the training dataset.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B1qxAIA5qvpR"
   },
   "source": [
    "Let's first create a batch prediction job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "CIujU9oBqwPp",
    "outputId": "4d9e7b9d-d8d6-44da-f178-5cd64f26d091",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_bp_input_1.jsonl [Content-Type=application/octet-stream]...\n",
      "/ [1 files][223.9 KiB/223.9 KiB]                                                \n",
      "Operation completed over 1 objects/223.9 KiB.                                    \n"
     ]
    }
   ],
   "source": [
    "BP_INPUT_URI_1 = f\"{BUCKET_URI}/model-monitoring/churn/churn_bp_input_1.jsonl\"\n",
    "! gsutil copy gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_bp_input_1.jsonl $BP_INPUT_URI_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-ZYgqB8ZmbDo",
    "outputId": "edc0cef9-0540-4ae1-d97a-60edc402227b",
    "tags": []
   },
   "outputs": [],
   "source": [
    "batch_prediction_job_1 = model.batch_predict(\n",
    "    generate_explanation=True,\n",
    "    job_display_name=\"bp_example_1\",\n",
    "    instances_format=\"jsonl\",\n",
    "    machine_type=\"e2-standard-4\",\n",
    "    gcs_source=[BP_INPUT_URI_1],\n",
    "    gcs_destination_prefix=f\"{BUCKET_URI}/bp_output\",\n",
    "    sync=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "S-3ilL5-03k8",
    "outputId": "85d46f2c-3351-4d75-e2c0-0ad6bd345d41",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Creating ModelMonitoringJob\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob created. Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:To use this ModelMonitoringJob in another session:\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:model_monitoring_job = aiplatform.ModelMonitoringJob('projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040')\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "TIMESTAMP = pd.Timestamp.utcnow().strftime(\"%Y%m%d%H%M%S\")\n",
    "JOB_DISPLAY_NAME = f\"churn_model_monitoring_job_{TIMESTAMP}\"\n",
    "TARGET_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    batch_prediction_job=batch_prediction_job_1.resource_name\n",
    ")\n",
    "model_monitoring_job_1 = my_model_monitor.run(\n",
    "    display_name=JOB_DISPLAY_NAME,\n",
    "    baseline_dataset=TRAINING_DATASET,\n",
    "    target_dataset=TARGET_DATASET,\n",
    "    tabular_objective_spec=ml_monitoring.spec.TabularObjective(\n",
    "        # Input feature drift spec.\n",
    "        feature_drift_spec=FEATURE_DRIFT_SPEC\n",
    "    ),\n",
    "    notification_spec=NOTIFICATION_SPEC,\n",
    "    output_spec=OUTPUT_SPEC,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ihfK5KxgWEXz"
   },
   "source": [
    "##### **Example 2: Detect feature drift and prediction output drift by comparing a batch prediction job with a previous batch prediction job result.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kBhhRjuIa2iv"
   },
   "source": [
    "You can set up multiple objectives within a single model monitoring job. All metrics are computed by using the same baseline and target dataset.\n",
    "\n",
    "Create another batch prediction job and compare it with the batch prediction job created previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IO2sXgSAXu5A",
    "outputId": "96682e51-daad-4eb0-fcdc-955f9990da32",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_bp_input_2.jsonl [Content-Type=application/octet-stream]...\n",
      "/ [1 files][443.4 KiB/443.4 KiB]                                                \n",
      "Operation completed over 1 objects/443.4 KiB.                                    \n"
     ]
    }
   ],
   "source": [
    "BP_INPUT_URI_2 = f\"{BUCKET_URI}/model-monitoring/churn/churn_bp_input_2.jsonl\"\n",
    "! gsutil copy gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_bp_input_2.jsonl $BP_INPUT_URI_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "d6CIK_1rWOT7",
    "outputId": "e7f70ea4-64c0-4080-83c2-d070305bd886",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob run completed. Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040\n"
     ]
    }
   ],
   "source": [
    "batch_prediction_job_2 = model.batch_predict(\n",
    "    job_display_name=\"bp_example_2\",\n",
    "    instances_format=\"jsonl\",\n",
    "    machine_type=\"e2-standard-4\",\n",
    "    gcs_source=[BP_INPUT_URI_2],\n",
    "    gcs_destination_prefix=f\"{BUCKET_URI}/bp_output\",\n",
    "    sync=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Knp5-4I-WKOa",
    "outputId": "a39ec128-135b-473b-aed8-d818f313971c",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Creating ModelMonitoringJob\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob created. Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:To use this ModelMonitoringJob in another session:\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:model_monitoring_job = aiplatform.ModelMonitoringJob('projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408')\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "TIMESTAMP = pd.Timestamp.utcnow().strftime(\"%Y%m%d%H%M%S\")\n",
    "JOB_DISPLAY_NAME = f\"churn_model_monitoring_job_{TIMESTAMP}\"\n",
    "BASELINE_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    batch_prediction_job=batch_prediction_job_1.resource_name\n",
    ")\n",
    "TARGET_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    batch_prediction_job=batch_prediction_job_2.resource_name\n",
    ")\n",
    "model_monitoring_job_2 = my_model_monitor.run(\n",
    "    display_name=JOB_DISPLAY_NAME,\n",
    "    baseline_dataset=BASELINE_DATASET,\n",
    "    target_dataset=TARGET_DATASET,\n",
    "    tabular_objective_spec=ml_monitoring.spec.TabularObjective(\n",
    "        # Input feature drift spec.\n",
    "        feature_drift_spec=FEATURE_DRIFT_SPEC,\n",
    "        # Prediction output drift spec.\n",
    "        prediction_output_drift_spec=PREDICTION_OUTPUT_DRIFT_SPEC,\n",
    "    ),\n",
    "    notification_spec=NOTIFICATION_SPEC,\n",
    "    output_spec=OUTPUT_SPEC,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AprOJ3ZchX7z"
   },
   "source": [
    "##### **Example 3: Feature attribution drift detection, compares the batch prediction job with a GCS baseline dataset**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kP7t8p8Ih2a-"
   },
   "source": [
    "For feature attribution monitoring, the dataset is sent to the Vertex AI batch explanation job in the following way:\n",
    "\n",
    "*   Google Cloud Storage -> Sent directly as input to Vertex AI batch explanation job.\n",
    "*   BigQuery table -> Sent directly as input to Vertex AI batch explanation job.\n",
    "*   BigQuery Query -> Not supported.\n",
    "*   Vertex AI batch explanation job -> Input of batch prediction job is used as input for the Vertex AI batch explanation job.\n",
    "*   Vertex AI endpoint logging -> Request logging is used as input for Vertex AI batch explanation job.\n",
    "\n",
    "Check that these datasets meet the requirements for a Vertex AI batch explanation job."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vmT9aF-YxGBS"
   },
   "source": [
    "###### Generate model metadata for Vertex Explainable AI\n",
    "You must specify the explanation specification to use a Vertex AI batch explanation job. Run the following cell to extract metadata from the exported model, which is needed for generating the explanations for a prediction request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "oLl6q8f4xFK5",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "from google.cloud.aiplatform_v1beta1.types import (ExplanationMetadata,\n",
    "                                                   ExplanationParameters,\n",
    "                                                   ExplanationSpec)\n",
    "\n",
    "EXPLANATION_SPEC = ExplanationSpec(\n",
    "    parameters=ExplanationParameters(\n",
    "        {\"sampled_shapley_attribution\": {\"path_count\": 2}}\n",
    "    ),\n",
    "    metadata=ExplanationMetadata(\n",
    "        inputs={\n",
    "            \"cnt_ad_reward\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_ad_reward\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_challenge_a_friend\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_challenge_a_friend\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_completed_5_levels\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_completed_5_levels\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_level_complete_quickplay\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_level_complete_quickplay\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_level_end_quickplay\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_level_end_quickplay\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_level_reset_quickplay\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_level_reset_quickplay\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_level_start_quickplay\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_level_start_quickplay\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_post_score\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_post_score\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_spend_virtual_currency\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_spend_virtual_currency\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_use_extra_steps\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_use_extra_steps\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"cnt_user_engagement\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"cnt_user_engagement\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"country\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"country\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"categorical\",\n",
    "                }\n",
    "            ),\n",
    "            \"dayofweek\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"dayofweek\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"julianday\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"julianday\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"language\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"language\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"categorical\",\n",
    "                }\n",
    "            ),\n",
    "            \"month\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"month\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"numeric\",\n",
    "                }\n",
    "            ),\n",
    "            \"operating_system\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"operating_system\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"categorical\",\n",
    "                }\n",
    "            ),\n",
    "            \"user_pseudo_id\": ExplanationMetadata.InputMetadata(\n",
    "                {\n",
    "                    \"input_tensor_name\": \"user_pseudo_id\",\n",
    "                    \"encoding\": \"IDENTITY\",\n",
    "                    \"modality\": \"categorical\",\n",
    "                }\n",
    "            ),\n",
    "        },\n",
    "        outputs={\n",
    "            \"churned_probs\": ExplanationMetadata.OutputMetadata(\n",
    "                {\"output_tensor_name\": \"churned_probs\"}\n",
    "            )\n",
    "        },\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hexssbF7wC6A",
    "outputId": "aed7dafb-4dac-41e6-9c01-7457689b3887",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Copying gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_no_ground_truth.jsonl [Content-Type=application/octet-stream]...\n",
      "/ [1 files][  3.5 MiB/  3.5 MiB]                                                \n",
      "Operation completed over 1 objects/3.5 MiB.                                      \n"
     ]
    }
   ],
   "source": [
    "FEATURE_ATTRIBUTION_BASELINE_DATASET = (\n",
    "    f\"{BUCKET_URI}/model-monitoring/churn/churn_no_ground_truth.jsonl\"\n",
    ")\n",
    "! gsutil cp gs://cloud-samples-data/vertex-ai/model-monitoring/churn/churn_no_ground_truth.jsonl $FEATURE_ATTRIBUTION_BASELINE_DATASET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2SJU52cymulD",
    "outputId": "6400ffb4-67e4-4d34-9c19-bd432af897ae",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Creating ModelMonitoringJob\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob created. Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:To use this ModelMonitoringJob in another session:\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:model_monitoring_job = aiplatform.ModelMonitoringJob('projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872')\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from vertexai.resources.preview import ml_monitoring\n",
    "\n",
    "TIMESTAMP = pd.Timestamp.utcnow().strftime(\"%Y%m%d%H%M%S\")\n",
    "JOB_DISPLAY_NAME = f\"churn_model_monitoring_job_{TIMESTAMP}\"\n",
    "BASELINE_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    gcs_uri=FEATURE_ATTRIBUTION_BASELINE_DATASET, data_format=\"jsonl\"\n",
    ")\n",
    "TARGET_DATASET = ml_monitoring.spec.MonitoringInput(\n",
    "    batch_prediction_job=batch_prediction_job_2.resource_name\n",
    ")\n",
    "model_monitoring_job_3 = my_model_monitor.run(\n",
    "    display_name=JOB_DISPLAY_NAME,\n",
    "    baseline_dataset=BASELINE_DATASET,\n",
    "    target_dataset=TARGET_DATASET,\n",
    "    tabular_objective_spec=ml_monitoring.spec.TabularObjective(\n",
    "        # Feature attribution spec.\n",
    "        feature_attribution_spec=FEATURE_ATTRIBUTION_SPEC\n",
    "    ),\n",
    "    # You must have a Explanation spec for feature attribution monitoring.\n",
    "    # You can specify the explanation spec in the Model, Model monitor, or the Model monitoring job.\n",
    "    explanation_spec=EXPLANATION_SPEC,\n",
    "    notification_spec=NOTIFICATION_SPEC,\n",
    "    output_spec=OUTPUT_SPEC,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pzYcMoKTP0rG"
   },
   "source": [
    "##### List Model Monitoring Jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "urFYUjaZ7iCP",
    "outputId": "e2681479-479b-43f4-f176-4cdb04c02620",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[name: \"projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872\"\n",
       "display_name: \"churn_model_monitoring_job_20251015150340\"\n",
       "model_monitoring_spec {\n",
       "  objective_spec {\n",
       "    tabular_objective {\n",
       "      feature_attribution_spec {\n",
       "        default_alert_condition {\n",
       "          threshold: 0.0003\n",
       "        }\n",
       "        feature_alert_conditions {\n",
       "          key: \"cnt_ad_reward\"\n",
       "          value {\n",
       "            threshold: 0.0001\n",
       "          }\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    explanation_spec {\n",
       "      parameters {\n",
       "        sampled_shapley_attribution {\n",
       "          path_count: 2\n",
       "        }\n",
       "      }\n",
       "      metadata {\n",
       "        inputs {\n",
       "          key: \"cnt_ad_reward\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_ad_reward\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_challenge_a_friend\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_challenge_a_friend\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_completed_5_levels\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_completed_5_levels\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_level_complete_quickplay\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_level_complete_quickplay\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_level_end_quickplay\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_level_end_quickplay\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_level_reset_quickplay\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_level_reset_quickplay\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_level_start_quickplay\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_level_start_quickplay\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_post_score\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_post_score\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_spend_virtual_currency\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_spend_virtual_currency\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_use_extra_steps\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_use_extra_steps\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"cnt_user_engagement\"\n",
       "          value {\n",
       "            input_tensor_name: \"cnt_user_engagement\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"country\"\n",
       "          value {\n",
       "            input_tensor_name: \"country\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"categorical\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"dayofweek\"\n",
       "          value {\n",
       "            input_tensor_name: \"dayofweek\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"julianday\"\n",
       "          value {\n",
       "            input_tensor_name: \"julianday\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"language\"\n",
       "          value {\n",
       "            input_tensor_name: \"language\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"categorical\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"month\"\n",
       "          value {\n",
       "            input_tensor_name: \"month\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"numeric\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"operating_system\"\n",
       "          value {\n",
       "            input_tensor_name: \"operating_system\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"categorical\"\n",
       "          }\n",
       "        }\n",
       "        inputs {\n",
       "          key: \"user_pseudo_id\"\n",
       "          value {\n",
       "            input_tensor_name: \"user_pseudo_id\"\n",
       "            encoding: IDENTITY\n",
       "            modality: \"categorical\"\n",
       "          }\n",
       "        }\n",
       "        outputs {\n",
       "          key: \"churned_probs\"\n",
       "          value {\n",
       "            output_tensor_name: \"churned_probs\"\n",
       "          }\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    baseline_dataset {\n",
       "      columnized_dataset {\n",
       "        gcs_source {\n",
       "          gcs_uri: \"gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/model-monitoring/churn/churn_no_ground_truth.jsonl\"\n",
       "          format_: JSONL\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    target_dataset {\n",
       "      batch_prediction_output {\n",
       "        batch_prediction_job: \"projects/684754974539/locations/us-central1/batchPredictionJobs/2302591152034938880\"\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  notification_spec {\n",
       "    email_config {\n",
       "      user_emails: \"stripling@google.com\"\n",
       "    }\n",
       "  }\n",
       "  output_spec {\n",
       "    gcs_base_directory {\n",
       "      output_uri_prefix: \"gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/model_monitoring/2411541261015056384\"\n",
       "    }\n",
       "  }\n",
       "}\n",
       "create_time {\n",
       "  seconds: 1760540620\n",
       "  nanos: 646554000\n",
       "}\n",
       "update_time {\n",
       "  seconds: 1760540620\n",
       "  nanos: 646554000\n",
       "}\n",
       "state: JOB_STATE_RUNNING\n",
       ", name: \"projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408\"\n",
       "display_name: \"churn_model_monitoring_job_20251015150307\"\n",
       "model_monitoring_spec {\n",
       "  objective_spec {\n",
       "    tabular_objective {\n",
       "      feature_drift_spec {\n",
       "        categorical_metric_type: \"l_infinity\"\n",
       "        numeric_metric_type: \"jensen_shannon_divergence\"\n",
       "        default_categorical_alert_condition {\n",
       "          threshold: 0.2\n",
       "        }\n",
       "        default_numeric_alert_condition {\n",
       "          threshold: 0.3\n",
       "        }\n",
       "        feature_alert_conditions {\n",
       "          key: \"cnt_user_engagement\"\n",
       "          value {\n",
       "            threshold: 0.004\n",
       "          }\n",
       "        }\n",
       "        feature_alert_conditions {\n",
       "          key: \"country\"\n",
       "          value {\n",
       "            threshold: 0.003\n",
       "          }\n",
       "        }\n",
       "      }\n",
       "      prediction_output_drift_spec {\n",
       "        categorical_metric_type: \"l_infinity\"\n",
       "        numeric_metric_type: \"jensen_shannon_divergence\"\n",
       "        default_categorical_alert_condition {\n",
       "          threshold: 0.1\n",
       "        }\n",
       "        default_numeric_alert_condition {\n",
       "          threshold: 0.1\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    baseline_dataset {\n",
       "      batch_prediction_output {\n",
       "        batch_prediction_job: \"projects/684754974539/locations/us-central1/batchPredictionJobs/2901640271219392512\"\n",
       "      }\n",
       "    }\n",
       "    target_dataset {\n",
       "      batch_prediction_output {\n",
       "        batch_prediction_job: \"projects/684754974539/locations/us-central1/batchPredictionJobs/2302591152034938880\"\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  notification_spec {\n",
       "    email_config {\n",
       "      user_emails: \"stripling@google.com\"\n",
       "    }\n",
       "  }\n",
       "  output_spec {\n",
       "    gcs_base_directory {\n",
       "      output_uri_prefix: \"gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/model_monitoring/2411541261015056384\"\n",
       "    }\n",
       "  }\n",
       "}\n",
       "create_time {\n",
       "  seconds: 1760540587\n",
       "  nanos: 726033000\n",
       "}\n",
       "update_time {\n",
       "  seconds: 1760540587\n",
       "  nanos: 726033000\n",
       "}\n",
       "state: JOB_STATE_RUNNING\n",
       ", name: \"projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8595201252697047040\"\n",
       "display_name: \"churn_model_monitoring_job_20251015144845\"\n",
       "model_monitoring_spec {\n",
       "  objective_spec {\n",
       "    tabular_objective {\n",
       "      feature_drift_spec {\n",
       "        categorical_metric_type: \"l_infinity\"\n",
       "        numeric_metric_type: \"jensen_shannon_divergence\"\n",
       "        default_categorical_alert_condition {\n",
       "          threshold: 0.2\n",
       "        }\n",
       "        default_numeric_alert_condition {\n",
       "          threshold: 0.3\n",
       "        }\n",
       "        feature_alert_conditions {\n",
       "          key: \"cnt_user_engagement\"\n",
       "          value {\n",
       "            threshold: 0.004\n",
       "          }\n",
       "        }\n",
       "        feature_alert_conditions {\n",
       "          key: \"country\"\n",
       "          value {\n",
       "            threshold: 0.003\n",
       "          }\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    baseline_dataset {\n",
       "      columnized_dataset {\n",
       "        gcs_source {\n",
       "          gcs_uri: \"gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/model-monitoring/churn/churn_training.csv\"\n",
       "          format_: CSV\n",
       "        }\n",
       "      }\n",
       "    }\n",
       "    target_dataset {\n",
       "      batch_prediction_output {\n",
       "        batch_prediction_job: \"projects/684754974539/locations/us-central1/batchPredictionJobs/2901640271219392512\"\n",
       "      }\n",
       "    }\n",
       "  }\n",
       "  notification_spec {\n",
       "    email_config {\n",
       "      user_emails: \"stripling@google.com\"\n",
       "    }\n",
       "  }\n",
       "  output_spec {\n",
       "    gcs_base_directory {\n",
       "      output_uri_prefix: \"gs://your-bucket-name-qwiklabs-gcp-01-89e8598a3363-unique/model_monitoring/2411541261015056384\"\n",
       "    }\n",
       "  }\n",
       "}\n",
       "create_time {\n",
       "  seconds: 1760539725\n",
       "  nanos: 444282000\n",
       "}\n",
       "update_time {\n",
       "  seconds: 1760539932\n",
       "  nanos: 935180000\n",
       "}\n",
       "state: JOB_STATE_SUCCEEDED\n",
       "job_execution_detail {\n",
       "  objective_status {\n",
       "    key: \"RAW_FEATURE_DRIFT\"\n",
       "    value {\n",
       "    }\n",
       "  }\n",
       "}\n",
       "]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "my_model_monitor.list_jobs()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nL10YABxl47v"
   },
   "source": [
    "### Step 5: Wait for the Model Monitoring Job to run and verify the result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LrSU6d0xFfzv"
   },
   "source": [
    "#### Verify results through email"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zw5KodgebDVE"
   },
   "source": [
    "After the model monitoring job begins running, which starts after the batch prediction jobs have finished, you receive an email like the following one:\n",
    "\n",
    "<img src=\"https://services.google.com/fh/files/misc/create_job_email.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "37SPm_majLVj"
   },
   "source": [
    "After the monitoring job is complete, if any anomalies are detected, you receive an email similar to the following one:\n",
    "\n",
    "<img src=\"https://services.google.com/fh/files/misc/job_anomalies_email.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LPC6ZoerifM6"
   },
   "source": [
    "#### Check monitoring metrics: Google Cloud Console"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4Pw6Z-bVbZaE"
   },
   "source": [
    "To view Model Monitoring metrics in the [Google Cloud Console](https://console.cloud.google.com/vertex-ai/model-monitoring/model-monitors), go to the **Monitoring** tab under **Vertex AI.**\n",
    "\n",
    "<img src=\"https://storage.googleapis.com/cmm-public-data/images/bp_details.gif\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MFZcBnwsdSfa"
   },
   "source": [
    "#### Check monitoring metrics: Cloud Storage bucket\n",
    "\n",
    "Run the following to view Model Monitoring metrics stored in the Cloud Storage bucket.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "NhY20BTjjkVk",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe id='facets-iframe' width=\"100%\" height=\"500px\"></iframe>\n",
       "        <script>\n",
       "        facets_iframe = document.getElementById('facets-iframe');\n",
       "        facets_html = '<script src=\"https://cdnjs.cloudflare.com/ajax/libs/webcomponentsjs/1.3.3/webcomponents-lite.js\"><\\/script><link rel=\"import\" href=\"https://raw.githubusercontent.com/PAIR-code/facets/master/facets-dist/facets-jupyter.html\"><facets-overview proto-input=\"CoSSAQoOQmFzZWxpbmUgU3RhdHMQ9AMauQYapQYKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEXsUrkfhetQ/GaWJVZtx+d8/INwCOQAAAAAAAABAQpkCGhIRmpmZmZmZyT8h5ZmZmZnBdUAaGwmamZmZmZnJPxGamZmZmZnZPyGamZmZmZm5PxobCZqZmZmZmdk/ETQzMzMzM+M/IZyZmZmZmbk/GhsJNDMzMzMz4z8RmpmZmZmZ6T8hmJmZmZmZuT8aGwmamZmZmZnpPxEAAAAAAADwPyEzMzMzM/NhQBobCQAAAAAAAPA/ETQzMzMzM/M/IR4YGBgYGLg/GhsJNDMzMzMz8z8RZ2ZmZmZm9j8hFhgYGBgYuD8aGwlnZmZmZmb2PxGamZmZmZn5PyEWGBgYGBi4PxobCZqZmZmZmfk/Ec3MzMzMzPw/IRYYGBgYGLg/GhsJzczMzMzM/D8RAAAAAAAAAEAhgH5+fn5+HkBCrwEaCSEAAAAAAABNQBoJIQAAAAAAAE1AGgkhAAAAAAAATUAaCSEAAAAAAABNQBoJIQAAAAAAAE1AGgkhAAAAAAAATUAaEhEAAAAAAADwPyEAAAAAAABIQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAEhAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASEAaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAAAgQCABQg8KDWNudF9hZF9yZXdhcmQaiwYa7gUKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEXsUrkfheqQ/GW5TfL0wFck/IOADOQAAAAAAAPA/QpkCGhIRmpmZmZmZuT8hd3wMzscAfkAaGwmamZmZmZm5PxGamZmZmZnJPyH6GJyPwfmoPxobCZqZmZmZmck/ETQzMzMzM9M/IfwYnI/B+ag/GhsJNDMzMzMz0z8RmpmZmZmZ2T8h+Bicj8H5qD8aGwmamZmZmZnZPxEAAAAAAADgPyH4GJyPwfmoPxobCQAAAAAAAOA/ETQzMzMzM+M/IQAZnI/B+ag/GhsJNDMzMzMz4z8RZ2ZmZmZm5j8h+Bicj8H5qD8aGwlnZmZmZmbmPxGamZmZmZnpPyH4GJyPwfmoPxobCZqZmZmZmek/Ec3MzMzMzOw/IfgYnI/B+ag/GhsJzczMzMzM7D8RAAAAAAAA8D8hk8H5GJyPM0BCeRoJIauqqqqqqkpAGgkhq6qqqqqqSkAaCSGrqqqqqqpKQBoJIauqqqqqqkpAGgkhq6qqqqqqSkAaCSGrqqqqqqpKQBoJIauqqqqqqkpAGgkhq6qqqqqqSkAaCSGrqqqqqqpKQBoSEQAAAAAAAPA/IQAAAAAAADRAIAFCGAoWY250X2NoYWxsZW5nZV9hX2ZyaWVuZBqeBhqBBgq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMREoPAyqFFxj8Zuzeha1RD2D8gnQM5AAAAAAAA8D9CmQIaEhGamZmZmZm5PyGQ9DShy9B5QBobCZqZmZmZmbk/EZqZmZmZmck/IfHekp4mdKk/GhsJmpmZmZmZyT8RNDMzMzMz0z8h896SniZ0qT8aGwk0MzMzMzPTPxGamZmZmZnZPyHv3pKeJnSpPxobCZqZmZmZmdk/EQAAAAAAAOA/Ie/ekp4mdKk/GhsJAAAAAAAA4D8RNDMzMzMz4z8h996SniZ0qT8aGwk0MzMzMzPjPxFnZmZmZmbmPyHv3pKeJnSpPxobCWdmZmZmZuY/EZqZmZmZmek/Ie/ekp4mdKk/GhsJmpmZmZmZ6T8RzczMzMzM7D8h796SniZ0qT8aGwnNzMzMzMzsPxEAAAAAAADwPyHLmo1UXaNVQEKLARoJIQAAAAAA0ElAGgkhAAAAAADQSUAaCSEAAAAAANBJQBoJIQAAAAAA0ElAGgkhAAAAAADQSUAaCSEAAAAAANBJQBoJIQAAAAAA0ElAGgkhAAAAAADQSUAaEhEAAAAAAADwPyEAAAAAAMBFQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAwEVAIAFCGAoWY250X2NvbXBsZXRlZF81X2xldmVscxr1BhrSBgq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMRtMh2vp8aFEAZ1NoJhhpVGkAg6wExAAAAAAAA8D85AAAAAAAAP0BCmQIaEhHNzMzMzMwIQCHdzMzMzOBxQBobCc3MzMzMzAhAEc3MzMzMzBhAIWdmZmZmhktAGhsJzczMzMzMGEARmpmZmZmZIkAhZmZmZmaGREAaGwmamZmZmZkiQBHNzMzMzMwoQCFnZmZmZoZDQBobCc3MzMzMzChAEQAAAAAAAC9AIWZmZmZmhkFAGhsJAAAAAAAAL0ARmpmZmZmZMkAhzczMzMwMMEAaGwmamZmZmZkyQBEzMzMzM7M1QCHMzMzMzAwwQBobCTMzMzMzszVAEc3MzMzMzDhAITQzMzMzMxRAGhsJzczMzMzMOEARZ2ZmZmbmO0AhzczMzMxMFEAaGwlnZmZmZuY7QBEAAAAAAAA/QCFjZmZmZmb4P0LTARoJIQAAAAAAYE1AGgkhAAAAAABgTUAaCSEAAAAAAGBNQBoJIQAAAAAAYE1AGhIRAAAAAAAA8D8hAAAAAAAAMEAaGwkAAAAAAADwPxEAAAAAAAAUQCEAAAAAAMBRQBobCQAAAAAAABRAEQAAAAAAABxAIQAAAAAAgEFAGhsJAAAAAAAAHEARAAAAAAAAJkAhAAAAAAAASUAaGwkAAAAAAAAmQBEAAAAAAAAuQCEAAAAAAIBIQBobCQAAAAAAAC5AEQAAAAAAAD9AIQAAAAAAAEZAIAFCHgocY250X2xldmVsX2NvbXBsZXRlX3F1aWNrcGxheRrwBhrSBgq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMRN4lBYOXQIUAZdq16RJgGJkAgzwExAAAAAAAAEEA5AAAAAACASUBCmQIaEhFmZmZmZmYUQCGtzMzMzCBxQBobCWZmZmZmZhRAEWZmZmZmZiRAIWdmZmZmBkpAGhsJZmZmZmZmJEARmZmZmZmZLkAhZmZmZmYGR0AaGwmZmZmZmZkuQBFmZmZmZmY0QCFmZmZmZgZGQBobCWZmZmZmZjRAEQAAAAAAgDlAIWZmZmZmhkFAGhsJAAAAAACAOUARmZmZmZmZPkAhzMzMzMwMM0AaGwmZmZmZmZk+QBGZmZmZmdlBQCGZmZmZmRkqQBobCZmZmZmZ2UFAEWZmZmZmZkRAIQEAAAAAgClAGhsJZmZmZmZmREARMzMzMzPzRkAhAAAAAAAA9j8aGwkzMzMzM/NGQBEAAAAAAIBJQCE0MzMzMzMEQELTARoJIQAAAAAA4ElAGgkhAAAAAADgSUAaCSEAAAAAAOBJQBoJIQAAAAAA4ElAGhIRAAAAAAAAEEAhAAAAAAAAR0AaGwkAAAAAAAAQQBEAAAAAAAAgQCEAAAAAAABKQBobCQAAAAAAACBAEQAAAAAAACpAIQAAAAAAAEhAGhsJAAAAAAAAKkARAAAAAAAAMkAhAAAAAAAAS0AaGwkAAAAAAAAyQBEAAAAAAAA5QCEAAAAAAABGQBobCQAAAAAAADlAEQAAAAAAgElAIQAAAAAAgEhAIAFCGQoXY250X2xldmVsX2VuZF9xdWlja3BsYXka8gYa0gYKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEV66SQwCKyBAGdqRRN5ZfSdAIPUBMQAAAAAAAPA/OQAAAAAAQFBAQpkCGhIRAAAAAAAAGkAhJwAAAAB0c0AaGwkAAAAAAAAaQBEAAAAAAAAqQCEAAAAAAOBOQBobCQAAAAAAACpAEQAAAAAAgDNAIQAAAAAAIERAGhsJAAAAAACAM0ARAAAAAAAAOkAhAAAAAABgREAaGwkAAAAAAAA6QBEAAAAAAEBAQCEAAAAAAEA0QBobCQAAAAAAQEBAEQAAAAAAgENAIQAAAAAAgCtAGhsJAAAAAACAQ0ARAAAAAADARkAhAAAAAACAGUAaGwkAAAAAAMBGQBEAAAAAAABKQCHNzMzMzMz+PxobCQAAAAAAAEpAEQAAAAAAQE1AIaZcyqVcygVAGhsJAAAAAABATUARAAAAAABAUEAhzvM8z/M87z9C0wEaCSEAAAAAAKBOQBoJIQAAAAAAoE5AGgkhAAAAAACgTkAaCSEAAAAAAKBOQBoSEQAAAAAAAPA/IQAAAAAAACZAGhsJAAAAAAAA8D8RAAAAAAAAFEAhAAAAAACASEAaGwkAAAAAAAAUQBEAAAAAAAAmQCEAAAAAAABIQBobCQAAAAAAACZAEQAAAAAAADFAIQAAAAAAAElAGhsJAAAAAAAAMUARAAAAAAAAOUAhAAAAAAAASEAaGwkAAAAAAAA5QBEAAAAAAEBQQCEAAAAAAIBIQCABQhsKGWNudF9sZXZlbF9yZXNldF9xdWlja3BsYXkahAca5AYKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEb6fGi/dZDBAGTvHmeuL3zJAIL0BMQAAAAAAACRAOQAAAAAAAFZAQpkCGhIRmpmZmZmZIUAh0MzMzMyMbkAaGwmamZmZmZkhQBGamZmZmZkxQCE0MzMzM3NMQBobCZqZmZmZmTFAEWdmZmZmZjpAITQzMzMzc01AGhsJZ2ZmZmZmOkARmpmZmZmZQUAhMzMzMzNzRkAaGwmamZmZmZlBQBEAAAAAAABGQCEzMzMzM3NFQBobCQAAAAAAAEZAEWdmZmZmZkpAIWdmZmZmZjpAGhsJZ2ZmZmZmSkARzszMzMzMTkAhAAAAAACAJ0AaGwnOzMzMzMxOQBGamZmZmZlRQCFmZmZmZuYlQBobCZqZmZmZmVFAEc3MzMzMzFNAIQnDMAzDMABAGhsJzczMzMzMU0ARAAAAAAAAVkAhmsAJnMAJ7D9C5QEaCSEAAAAAAIBPQBoJIQAAAAAAgE9AGgkhAAAAAACAT0AaEhEAAAAAAAAAQCEAAAAAAAAuQBobCQAAAAAAAABAEQAAAAAAACRAIQAAAAAAgEpAGhsJAAAAAAAAJEARAAAAAAAAMUAhAAAAAAAARkAaGwkAAAAAAAAxQBEAAAAAAAA5QCEAAAAAAIBKQBobCQAAAAAAADlAEQAAAAAAAEFAIQAAAAAAAEhAGhsJAAAAAAAAQUARAAAAAACARkAhAAAAAACASUAaGwkAAAAAAIBGQBEAAAAAAABWQCEAAAAAAIBHQCABQhsKGWNudF9sZXZlbF9zdGFydF9xdWlja3BsYXka+QYa5AYKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEe58PzVeOiBAGdTo/btOPiNAILgBMQAAAAAAABBAOQAAAAAAAEZAQpkCGhIRmpmZmZmZEUAhQTMzMzMTcEAaGwmamZmZmZkRQBGamZmZmZkhQCGZmZmZmZlJQBobCZqZmZmZmSFAEWdmZmZmZipAIczMzMzMbFBAGhsJZ2ZmZmZmKkARmpmZmZmZMUAhmpmZmZkZQEAaGwmamZmZmZkxQBEAAAAAAAA2QCGamZmZmVlCQBobCQAAAAAAADZAEWdmZmZmZjpAITQzMzMzMzdAGhsJZ2ZmZmZmOkARzszMzMzMPkAhMzMzMzMzNEAaGwnOzMzMzMw+QBGamZmZmZlBQCFDREREREQnQBobCZqZmZmZmUFAEc3MzMzMzENAITeO4ziO4/A/GhsJzczMzMzMQ0ARAAAAAAAARkAhfNInfdIn7T9C5QEaCSGrqqqqqqpOQBoJIauqqqqqqk5AGgkhq6qqqqqqTkAaEhEAAAAAAAAAQCEAAAAAAIBDQBobCQAAAAAAAABAEQAAAAAAABBAIQAAAAAAAEFAGhsJAAAAAAAAEEARAAAAAAAAIEAhAAAAAACASUAaGwkAAAAAAAAgQBEAAAAAAAAmQCEAAAAAAIBGQBobCQAAAAAAACZAEQAAAAAAADBAIQAAAAAAAEhAGhsJAAAAAAAAMEARAAAAAAAAN0AhAAAAAAAASkAaGwkAAAAAAAA3QBEAAAAAAABGQCEAAAAAAIBHQCABQhAKDmNudF9wb3N0X3Njb3JlGtgGGrcGCrYCCPQDGAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAgAUD0AxHAyqFFtvPtPxlvFtHz8iL0PyCPAjkAAAAAAAAYQEKZAhoSETMzMzMzM+M/IfTMzMzM9HBAGhsJMzMzMzMz4z8RMzMzMzMz8z8hMzMzMzNzVUAaGwkzMzMzMzPzPxHMzMzMzMz8PyEyMzMzMzPTPxobCczMzMzMzPw/ETMzMzMzMwNAITMzMzMzc1RAGhsJMzMzMzMzA0ARAAAAAAAACEAhZmZmZmZmQUAaGwkAAAAAAAAIQBHMzMzMzMwMQCEwMzMzMzPTPxobCczMzMzMzAxAEc3MzMzMzBBAIc3MzMzMzDJAGhsJzczMzMzMEEARMzMzMzMzE0AhMDMzMzMz0z8aGwkzMzMzMzMTQBGZmZmZmZkVQCEK16NwPQoTQBobCZmZmZmZmRVAEQAAAAAAABhAIXI9CtejcP0/QsEBGgkhmpmZmZkZS0AaCSGamZmZmRlLQBoJIZqZmZmZGUtAGgkhmpmZmZkZS0AaCSGamZmZmRlLQBoSEQAAAAAAAPA/IQAAAAAAgEVAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAACARUAaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAIBUQBobCQAAAAAAAABAEQAAAAAAAAhAIQAAAAAAgEFAGhsJAAAAAAAACEARAAAAAAAAGEAhAAAAAAAAOkAgAUIcChpjbnRfc3BlbmRfdmlydHVhbF9jdXJyZW5jeRrRBhq3Bgq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMREFg5tMh27j8ZmLnOX4fj8j8g/gE5AAAAAAAAFEBCmQIaEhEAAAAAAADgPyFbAAAAAMhvQBobCQAAAAAAAOA/EQAAAAAAAPA/IQAAAAAA8FhAGhsJAAAAAAAA8D8RAAAAAAAA+D8hAAAAAAAA0D8aGwkAAAAAAAD4PxEAAAAAAAAAQCEAAAAAADBUQBobCQAAAAAAAABAEQAAAAAAAARAIQAAAAAAANA/GhsJAAAAAAAABEARAAAAAAAACEAhAAAAAADgSEAaGwkAAAAAAAAIQBEAAAAAAAAMQCEAAAAAAADQPxobCQAAAAAAAAxAEQAAAAAAABBAIQAAAAAAgCVAGhsJAAAAAAAAEEARAAAAAAAAEkAhHMdxHMdxzD8aGwkAAAAAAAASQBEAAAAAAAAUQCGQ4ziO4zgOQELBARoJIWZmZmZmZklAGgkhZmZmZmZmSUAaCSFmZmZmZmZJQBoJIWZmZmZmZklAGgkhZmZmZmZmSUAaEhEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAAAEAhAAAAAABAVEAaGwkAAAAAAAAAQBEAAAAAAAAIQCEAAAAAAABJQBobCQAAAAAAAAhAEQAAAAAAABRAIQAAAAAAAC5AIAFCFQoTY250X3VzZV9leHRyYV9zdGVwcxqQBxr2Bgq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMRhutRuB4FREAZkV6PtncTREAghQExAAAAAAAAP0A5AAAAAACAaEBCmQIaEhGamZmZmZkzQCG1mZmZmUlpQBobCZqZmZmZmTNAEZqZmZmZmUNAITQzMzMzM1NAGhsJmpmZmZmZQ0ARZ2ZmZmZmTUAhNDMzMzMTU0AaGwlnZmZmZmZNQBGamZmZmZlTQCFmZmZmZmZLQBobCZqZmZmZmVNAEQAAAAAAgFhAIWZmZmZm5kNAGhsJAAAAAACAWEARZ2ZmZmZmXUAhNDMzMzNTOEAaGwlnZmZmZmZdQBFnZmZmZiZhQCHqoA7qoM4tQBobCWdmZmZmJmFAEZqZmZmZmWNAIVQacu6zCxhAGhsJmpmZmZmZY0ARzczMzMwMZkAh43oUrkfhCEAaGwnNzMzMzAxmQBEAAAAAAIBoQCFfZmZmZmb6P0L3ARoJIQAAAAAAoFBAGgkhAAAAAACgUEAaEhEAAAAAAAAYQCEAAAAAAAAyQBobCQAAAAAAABhAEQAAAAAAADNAIQAAAAAAgElAGhsJAAAAAAAAM0ARAAAAAAAAP0AhAAAAAAAASUAaGwkAAAAAAAA/QBEAAAAAAABGQCEAAAAAAABJQBobCQAAAAAAAEZAEQAAAAAAAE1AIQAAAAAAgEpAGhsJAAAAAAAATUARAAAAAADAUkAhAAAAAAAASEAaGwkAAAAAAMBSQBEAAAAAAABZQCEAAAAAAIBIQBobCQAAAAAAAFlAEQAAAAAAgGhAIQAAAAAAAEhAIAFCFQoTY250X3VzZXJfZW5nYWdlbWVudBrdBxACIs0HCrYCCPQDGAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAgAUD0AxATGhgSDVVuaXRlZCBTdGF0ZXMZAAAAAAAwc0AaEBIFSmFwYW4ZAAAAAAAAQEAaERIGQ2FuYWRhGQAAAAAAAEBAGhASBUluZGlhGQAAAAAAAD5AGhQSCUF1c3RyYWxpYRkAAAAAAAA2QBoZEg5Vbml0ZWQgS2luZ2RvbRkAAAAAAAA0QBoREgZNZXhpY28ZAAAAAAAAJEAaERIGVGFpd2FuGQAAAAAAACBAGhcSDFNvdXRoIEFmcmljYRkAAAAAAAAcQBoSEgdHZXJtYW55GQAAAAAAABxAGhESBkZyYW5jZRkAAAAAAAAUQBoWEgtOZXRoZXJsYW5kcxkAAAAAAAAQQBoQEgVDaGluYRkAAAAAAAAQQBoWEgtQaGlsaXBwaW5lcxkAAAAAAAAIQBoQEgVFZ3lwdBkAAAAAAAAIQBoREgZCcmF6aWwZAAAAAAAACEAaEhIHVmlldG5hbRkAAAAAAADwPxoXEgxTYXVkaSBBcmFiaWEZAAAAAAAA8D8aExIIUGFraXN0YW4ZAAAAAAAA8D8lnMQsQSr5AQoYIg1Vbml0ZWQgU3RhdGVzKQAAAAAAMHNAChQIARABIgVKYXBhbikAAAAAAABAQAoVCAIQAiIGQ2FuYWRhKQAAAAAAAEBAChQIAxADIgVJbmRpYSkAAAAAAAA+QAoYCAQQBCIJQXVzdHJhbGlhKQAAAAAAADZACh0IBRAFIg5Vbml0ZWQgS2luZ2RvbSkAAAAAAAA0QAoVCAYQBiIGTWV4aWNvKQAAAAAAACRAChUIBxAHIgZUYWl3YW4pAAAAAAAAIEAKGwgIEAgiDFNvdXRoIEFmcmljYSkAAAAAAAAcQAoWCAkQCSIHR2VybWFueSkAAAAAAAAcQEIJCgdjb3VudHJ5GsIHGrIHCrYCCPQDGAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAgAUD0AxFcj8L1KFwPQBm4jEnB5qL/PykAAAAAAADwPzEAAAAAAAAQQDkAAAAAAAAcQEKiAhobCQAAAAAAAPA/EZqZmZmZmfk/ITMzMzMzE1NAGhsJmpmZmZmZ+T8RmpmZmZmZAUAhMzMzMzNzUEAaGwmamZmZmZkBQBFmZmZmZmYGQCEwMzMzMzPTPxobCWZmZmZmZgZAETMzMzMzMwtAITMzMzMz81JAGhsJMzMzMzMzC0ARAAAAAAAAEEAhMzMzMzPzVUAaGwkAAAAAAAAQQBFmZmZmZmYSQCEwMzMzMzPTPxobCWZmZmZmZhJAEc3MzMzMzBRAIWdmZmZmZk1AGhsJzczMzMzMFEARMzMzMzMzF0AhMDMzMzMz0z8aGwkzMzMzMzMXQBGZmZmZmZkZQCFkkmrtGvNQQBobCZmZmZmZmRlAEQAAAAAAABxAIfXT+3hLs1BAQqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAAU0AaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAIBQQBobCQAAAAAAAABAEQAAAAAAAAhAIQAAAAAAAENAGhsJAAAAAAAACEARAAAAAAAACEAhAAAAAAAAQ0AaGwkAAAAAAAAIQBEAAAAAAAAQQCEAAAAAAABGQBobCQAAAAAAABBAEQAAAAAAABBAIQAAAAAAAEZAGhsJAAAAAAAAEEARAAAAAAAAFEAhAAAAAACATUAaGwkAAAAAAAAUQBEAAAAAAAAYQCEAAAAAAABRQBobCQAAAAAAABhAEQAAAAAAABxAIQAAAAAAwEBAGhsJAAAAAAAAHEARAAAAAAAAHEAhAAAAAADAQEAgAUILCglkYXlvZndlZWsawgcasgcKtgII9AMYASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQCABQPQDEdNNYhBYsWlAGQX0YklP3UBAKQAAAAAAwFpAMQAAAAAAoGlAOQAAAAAAIHNAQqICGhsJAAAAAADAWkARmpmZmZm5X0AhmpmZmZk5HEAaGwmamZmZmblfQBGamZmZmVliQCGbmZmZmckmQBobCZqZmZmZWWJAEWZmZmZm1mRAITIzMzMz80NAGhsJZmZmZmbWZEARMzMzMzNTZ0AhzczMzMy8VEAaGwkzMzMzM1NnQBEAAAAAANBpQCHNzMzMzLxfQBobCQAAAAAA0GlAEczMzMzMTGxAIczMzMzMPFdAGhsJzMzMzMxMbEARmZmZmZnJbkAhAAAAAAAIVEAaGwmZmZmZmcluQBEzMzMzM6NwQCHNzMzMzHxEQBobCTMzMzMzo3BAEZqZmZmZ4XFAIVhVVVVVFSpAGhsJmpmZmZnhcUARAAAAAAAgc0AhICIiIiKiEkBCpAIaGwkAAAAAAMBaQBEAAAAAAKBkQCEAAAAAAABKQBobCQAAAAAAoGRAEQAAAAAAYGZAIQAAAAAAgEpAGhsJAAAAAABgZkARAAAAAACAZ0AhAAAAAAAASUAaGwkAAAAAAIBnQBEAAAAAAIBoQCEAAAAAAIBIQBobCQAAAAAAgGhAEQAAAAAAoGlAIQAAAAAAAExAGhsJAAAAAACgaUARAAAAAADAakAhAAAAAACARUAaGwkAAAAAAMBqQBEAAAAAAABsQCEAAAAAAIBKQBobCQAAAAAAAGxAEQAAAAAAgG1AIQAAAAAAgEZAGhsJAAAAAACAbUARAAAAAAAgb0AhAAAAAAAASkAaGwkAAAAAACBvQBEAAAAAACBzQCEAAAAAAIBHQCABQgsKCWp1bGlhbmRheRrMAxACIr4DCrYCCPQDGAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAgAUD0AxAEGgwSATYZAAAAAAAga0AaDBIBORkAAAAAAEBaQBoMEgE3GQAAAAAAAFhAGgwSATgZAAAAAACAVEAlAACAPypECgwiATYpAAAAAAAga0AKEAgBEAEiATkpAAAAAABAWkAKEAgCEAIiATcpAAAAAAAAWEAKEAgDEAMiATgpAAAAAACAVEBCBwoFbW9udGgazQMQAiK0Awq2Agj0AxgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAIAFA9AMQAxoSEgdBTkRST0lEGQAAAAAAUHBAGg4SA0lPUxkAAAAAAGBrQBoPEgRudWxsGQAAAAAAADRAJZMYpEAqPQoSIgdBTkRST0lEKQAAAAAAUHBAChIIARABIgNJT1MpAAAAAABga0AKEwgCEAIiBG51bGwpAAAAAAAANEBCEgoQb3BlcmF0aW5nX3N5c3RlbRrVKRACIr4pCrYCCPQDGAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAaGwkAAAAAAADwPxEAAAAAAADwPyEAAAAAAABJQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAAElAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAAAASUAgAUD0AxD0AxorEiBGRkVFNzMwNzZDNTcwOTA2M0JDNDU0OTZDODQ2MzU5QhkAAAAAAADwPxorEiBGRkFBRTVFRTI5Q0RCODc2MDdFMzU2QTUyRTExOUY3NhkAAAAAAADwPxorEiBGRUFDNDhFNDcyNjc4RDFFMkU1NTY0RjY0MUJCNkI0NxkAAAAAAADwPxorEiBGRTgwMjgyMzhGQkU5N0VGQUE4NDY4QzJEQzNCQjM0MhkAAAAAAADwPxorEiBGRTUwOEI5Q0FFRDEyOEM0OTdFQ0UxOEQ3QzM3QUYxNBkAAAAAAADwPxorEiBGRENCRTNDQjk2OTU1NkQxRjJGMkFBN0JEOEYzRkE4RRkAAAAAAADwPxorEiBGRDYyRDFDNTcwMEY2Q0MwNzc4Q0VGQzM2N0Y3RjE3NRkAAAAAAADwPxorEiBGQzkxM0M1MjQ1MTc4QUY3QjMwQTgwOTIyMDlDMTZGQxkAAAAAAADwPxorEiBGQzQyMDRCMDZGODUxNjhBNDg3NDlDNjg3RTBFMjhFMhkAAAAAAADwPxorEiBGQkM3N0ZFRDdFOEJEQzJCM0JCOUFCRDI5N0E5NDM1MhkAAAAAAADwPxorEiBGQkE5OTQzNjQ0QUEzQkQ4RjA0QTdFMTUyQUEwOEE3NBkAAAAAAADwPxorEiBGQUZDRjYwMDkyNjA3OTg0REEyQzI2ODdGNDFDQUQ1QxkAAAAAAADwPxorEiBGQUU1Q0UwMDc2QkJDRTE1NUVDMzBCOTQ4MzIxNEQ2RBkAAAAAAADwPxorEiBGQTAxMDQxRENGMEM0MzYzNERDRDg5RjI0MEMzQjQ1NhkAAAAAAADwPxorEiBGOUVGQjZCREZCQ0NCOTE2N0YxMDRFNjM2MzgyNTk2ORkAAAAAAADwPxorEiBGOUE2NEJDODkzOEI4RUY0QUJDMUEyQzNGQkZFQkM4NhkAAAAAAADwPxorEiBGOTgwMUQ2OThDREZEM0RCNTdDQjlCNzZBMzdFMjQ4RRkAAAAAAADwPxorEiBGOTNEOTdEMjFCNTUzMDk3Q0VGRDZCRUI5NEM3QThBORkAAAAAAADwPxorEiBGODlFQ0VBODk1RUZFMjVFODY0QUI1NzIzNEVCNUUyMhkAAAAAAADwPxorEiBGNzQxMUQ2RTg5MkREMkE3MTBDMUJDOUY2QjQ4NTdENxkAAAAAAADwPxorEiBGNkY5NDYxRjcxMzFDNERFQzk2RTNBRkQwREIzODA5NBkAAAAAAADwPxorEiBGNTlDNDUwQzU3OEE0NUEzQkRBOEM1ODVFNjA0MTk3ORkAAAAAAADwPxorEiBGNTZDMjdCMTZGRTM3ODYyMzAxMTJBNTY4RDFERjZENhkAAAAAAADwPxorEiBGNTVBOEY1ODNCNjU0RTQ0MEVCMzQ2NUQ0QjM5RjgyNhkAAAAAAADwPxorEiBGNERDMTdGQzIwNDFCREFEMzcyMjgxMEZEMzQ1Qjk5QhkAAAAAAADwPxorEiBGNDgwOEJDQkExNTMxMEQ3Q0VGMjFEN0UzODVFOUQ0NRkAAAAAAADwPxorEiBGNDQ1RDQ5MzdDODRDQzU5OERGNERDRTUyNzVGREI5ORkAAAAAAADwPxorEiBGNDIzNTYwQzJDOEQ2MzYyMTFEODdERDY1MTMzMjVGRRkAAAAAAADwPxorEiBGMzc0QURCQkQ5MkUyRjhGQzQ0RDRBMDQ2REVGNUZBMhkAAAAAAADwPxorEiBGMzI3NjMwMDRBMjdBNjlBMDREQzQwQzk5NzNDQ0IwNBkAAAAAAADwPxorEiBGMkEzMzE1RUE5RTU4NDI2OTVCQkMxMDdCMTdFQzMwOBkAAAAAAADwPxorEiBGMTREQTJDRUYwOTYwN0JFQkZCMTMxMTU5RUI5QzRGMRkAAAAAAADwPxorEiBGMDlGRUJCMjhCRkEyRDU3RjlGNzQ2NkUyNDhBNTU2ORkAAAAAAADwPxorEiBFRjk5OUVERUI1QkUwOEE2RjlEQzAyREFFMUNBNDBDNBkAAAAAAADwPxorEiBFRTAzNTIzREIzNzY3RDVGQjQ0Nzk4NTlEOEQ5Mjk4NhkAAAAAAADwPxorEiBFQ0ZBN0I3RkNBOURGMzRDQ0E3NUIzN0NDMjkxM0QzMxkAAAAAAADwPxorEiBFQ0U1QkEwMzFFQUJCNERGMDA3OTRBMUZGNEUxNkU4OBkAAAAAAADwPxorEiBFQzY4OTIxNDFDQjQwMzgwRjgxMEY1MDREQzExRDI3MhkAAAAAAADwPxorEiBFQkVGMEZDMTUwRkMzQjNBOTUwQkFCMzQwMzZGRTkxMhkAAAAAAADwPxorEiBFQjk1MTg0MkIxMDBDOTgxQUQzQTlDRjBFOTFDMTAwMhkAAAAAAADwPxorEiBFQjkyQzlFQTgwOUIwQ0QxQUEyNjAwMTI0MzhENjAwORkAAAAAAADwPxorEiBFQTNDQjY3OTMzQjkwNEE1NjBFNzNCODgyQkQyQTk5MBkAAAAAAADwPxorEiBFOUU2REJGQjY0QjBEOEJCOEFDMDRGN0VFM0I3RTA0QhkAAAAAAADwPxorEiBFOUJBREIzMzA0OTlCMDkyNTcyOEUwQUQ4QTNDRUEwOBkAAAAAAADwPxorEiBFOTg2REExODdGNDM2NURCOTgwNjcwQkY2NDVBNkMwNBkAAAAAAADwPxorEiBFOTM0QTFENDhCNUU2MUQ3NjhGNEUzRkQ5NENGQ0Q0ORkAAAAAAADwPxorEiBFODJBQjVFNjc2NkIxMTgyNzc1OEQ0QTIxRjQxNzVDNhkAAAAAAADwPxorEiBFODAyNUIxMjMyREM4QjMxN0VDOUYwQkU1RjgyNDU0QRkAAAAAAADwPxorEiBFN0MxODVGMTQzNjlGNzg5RjUyNTczMDFGMUMwNjZEMBkAAAAAAADwPxorEiBFNzQ2OTJEQjJCODUyNjQ0ODA2RTdBMkI0OEJCRDY4QxkAAAAAAADwPxorEiBFNzM0RkMwNjIzMENDMjNBRDkwMjEzMTkxMjA5QTkxNBkAAAAAAADwPxorEiBFNTI1QUE5OTM1NzE4QjcwOEFFQTUzNkI2NEVGQzQwNxkAAAAAAADwPxorEiBFMzc0QUZCNDU0MEZFOUQyQUVFMkVDQTNBQUQzRTI3NRkAAAAAAADwPxorEiBFMzcwRkI3RDI3QjI1RTdFNjI0MTMzRjgxMEVFMDAyRhkAAAAAAADwPxorEiBFMzI4MTgyNDREOEYwMTk2MzhGNTE5RjAyNTQyMzA3OBkAAAAAAADwPxorEiBFMzIxMTNGOUY5QTM0RjhBRUI2M0QxNEIzQ0IxNTUyNxkAAAAAAADwPxorEiBFMzE4RTJBMDEwQkRBMUU4NEU2NURBMEE1MjUyMzQ4OBkAAAAAAADwPxorEiBFMjAwQUVBMzVFRjM1OTFGNzJFNEU3NEYzOTkyNkU4RhkAAAAAAADwPxorEiBFMUI1REM1MUE4NjcyRTJEMzkzREU2ODY3REQ5RTJEOBkAAAAAAADwPxorEiBFMTgzRDYzMzY0QjE0RTQwNkQzQkREMUEzNDVCNUNCRhkAAAAAAADwPxorEiBFMTMzM0JCRTUwQTNGRkVFN0ZEMjA2NkE2NkQyQTBEQxkAAAAAAADwPxorEiBFMEY5MDZCNjZDMzZDMDIzRkM0MTQ3Q0RENDJBMTRGQRkAAAAAAADwPxorEiBFMDdDRjFDQTg4QzJFMDIzNTE2RkY2QTBCNTc2MEYzNBkAAAAAAADwPxorEiBFMDQ2MTc1MEQ1Rjk0MEJEQzBGODhDOTRGQkQ1ODFGMhkAAAAAAADwPxorEiBERjc3NDk1MjAzRjQwRTZDQ0VGNzQ1RUE2NjIxMDkwRRkAAAAAAADwPxorEiBERUI5RDVEREE5OTM1NDQzMUYzMEQxNjc1QzREOTYzMRkAAAAAAADwPxorEiBERDkzMUFCREMyNkQ1NkZCQjBGRjk5NTQ2NzM3QjMyMhkAAAAAAADwPxorEiBEQzFBRjE5RkZGRDQ1RTdBRUMxREJDMjE3MzY2MEM2MhkAAAAAAADwPxorEiBEQzAwM0UyNzk2NTA4OEQ4QkJBQjI2OTJBRjQwRDFGNRkAAAAAAADwPxorEiBEQkY0NzhDRERENzA5RTIwMzRFNTA1OTA3ODlGOTE2MBkAAAAAAADwPxorEiBEQkVEM0EwMzZGNDM0OEU1MzExMjMxMjVDNEQxNzAwRhkAAAAAAADwPxorEiBEQjlCRTkyRTg2NTBBOUVCODc2NTIyQzgwNDg2NjEwRhkAAAAAAADwPxorEiBEQjRDNDgwNzg0NDNBOUVGNjEwQzkxRTc3RTFBQzUyMRkAAAAAAADwPxorEiBEQjI0OTE1NTA2NjQxQ0NDRkZGRTkzNEY2OEE0ODM2QhkAAAAAAADwPxorEiBEOUEzNTIxOTk5MDk5MkNGMUZENEU2MTgxM0QxRUZFQxkAAAAAAADwPxorEiBEOTg2OUFEOTlBQzc0M0VBMzkwRjE2NTA5NDhDQTIwQhkAAAAAAADwPxorEiBEOTU4QzdDOUY4RjYyQUVBQUQ4MjQ1NzlEMTY0MjEwQhkAAAAAAADwPxorEiBEOTQwNUJBNUY4NjBDNTgzRDgzNzFGNDUyOTg3Rjg1OBkAAAAAAADwPxorEiBEODhFMUVBMDYwMjU5MTZERDIwRUFBRkE4NUE2QjQwMhkAAAAAAADwPxorEiBEODdCN0ZCNjVEQUM4RkYwNDU1QUVFMjhGQUIxMjZEORkAAAAAAADwPxorEiBEODAwN0M5REZDQ0RDOTc5MDNDMjFCQzYwRDYzQUQxOBkAAAAAAADwPxorEiBEN0Y1QzIyMkU2M0IzNzk1QkRBOTc1QTAyNUMwN0U1ORkAAAAAAADwPxorEiBENUVCNUY2MTAzQTlEMzFGQjgwQ0UxNTI2RTdDMERCNRkAAAAAAADwPxorEiBENTJDMjREQkQyOUJBQ0Q5RDk5RUEwMzk0OUM4NEZGQhkAAAAAAADwPxorEiBENDVEMEMzNjM1RTU4NDBGNEU2RTBDQTk0MzlGNDM2MxkAAAAAAADwPxorEiBENDVCMUI5NEVENzZDNjY0OEMwNjk0RDBDNzRBOTZBQxkAAAAAAADwPxorEiBENDQ3MjNDMjM1M0EzN0JENTQzMzcyM0RFOTA3ODVDORkAAAAAAADwPxorEiBEMkFEMzZFMzhCMkJERDI0NzY0Mjk2RTJBNDU3RDEwMRkAAAAAAADwPxorEiBEMjcwMzQ3QkYyMzcyMTcyRERERDM3ODQ0Q0FGODZGQxkAAAAAAADwPxorEiBEMUFGNjI4RkQ1OEM0MTBDQkNBMzkwQ0Q4OThFRjc2MxkAAAAAAADwPxorEiBEMTY4N0E1RTk2M0Y1NzVGNTRFN0UyNTlGQURDQ0Y5MBkAAAAAAADwPxorEiBEMTAxMEQ2MjREQ0FGMzIxOEIxRkI1NzAyNzNBMzVDMRkAAAAAAADwPxorEiBEMEI1RTQ2RDVERTIyOTg3N0YzMTg2MDY0RTdENTM5RhkAAAAAAADwPxorEiBEMDg2QzQwOUVEMUUyMjkxRDY0OTc1MUFBN0QyRjk3MRkAAAAAAADwPxorEiBDRkZGMjQ2RTBDOURBMDgyRkVBRENFREZFN0RFNUI0RhkAAAAAAADwPxorEiBDRkVGREE4RTc2MTRDQkQ5QzU3QUIxRUM3Nzg5MzQ0RhkAAAAAAADwPxorEiBDRjRGQTI2QzdDNjdDN0NFM0VGNTlCRkY4NEI2RTExNRkAAAAAAADwPxorEiBDRUQ4QTRFODNCNTJDM0RGOUI2NzYwNkM3MEIwMjU5RBkAAAAAAADwPxorEiBDRTBBMDcxQ0IyRDlBRDc0MjQyQzkzMUJBNEQ2MzJDORkAAAAAAADwPxorEiBDREY0MTIyREY0OURCMEFFMzdFRTM3NEI2ODM4Mjg4MRkAAAAAAADwPyUAAABCKuYDCisiIEZGRUU3MzA3NkM1NzA5MDYzQkM0NTQ5NkM4NDYzNTlCKQAAAAAAAPA/Ci8IARABIiBGRkFBRTVFRTI5Q0RCODc2MDdFMzU2QTUyRTExOUY3NikAAAAAAADwPwovCAIQAiIgRkVBQzQ4RTQ3MjY3OEQxRTJFNTU2NEY2NDFCQjZCNDcpAAAAAAAA8D8KLwgDEAMiIEZFODAyODIzOEZCRTk3RUZBQTg0NjhDMkRDM0JCMzQyKQAAAAAAAPA/Ci8IBBAEIiBGRTUwOEI5Q0FFRDEyOEM0OTdFQ0UxOEQ3QzM3QUYxNCkAAAAAAADwPwovCAUQBSIgRkRDQkUzQ0I5Njk1NTZEMUYyRjJBQTdCRDhGM0ZBOEUpAAAAAAAA8D8KLwgGEAYiIEZENjJEMUM1NzAwRjZDQzA3NzhDRUZDMzY3RjdGMTc1KQAAAAAAAPA/Ci8IBxAHIiBGQzkxM0M1MjQ1MTc4QUY3QjMwQTgwOTIyMDlDMTZGQykAAAAAAADwPwovCAgQCCIgRkM0MjA0QjA2Rjg1MTY4QTQ4NzQ5QzY4N0UwRTI4RTIpAAAAAAAA8D8KLwgJEAkiIEZCQzc3RkVEN0U4QkRDMkIzQkI5QUJEMjk3QTk0MzUyKQAAAAAAAPA/QhAKDnVzZXJfcHNldWRvX2lkCoieAQoMVGFyZ2V0IFN0YXRzEN8+GoIGGu4FCrYCCN8+GAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAgAUDfPhHPGqkZBaCsPxl2Rr+kKkvkPyDAPTkAAAAAAAA9QEKZAhoSETMzMzMzMwdAIRaamZmZO79AGhsJMzMzMzMzB0ARMzMzMzMzF0AhZ2ZmZmZmMkAaGwkzMzMzMzMXQBFmZmZmZmYhQCEyMzMzMzMiQBobCWZmZmZmZiFAETMzMzMzMydAIf4oXI/C9fw/GhsJMzMzMzMzJ0ARAAAAAAAALUAhPgrXo3A98D8aGwkAAAAAAAAtQBFmZmZmZmYxQCE7CtejcD3wPxobCWZmZmZmZjFAEc3MzMzMTDRAIUEK16NwPfA/GhsJzczMzMxMNEARMzMzMzMzN0AhOwrXo3A98D8aGwkzMzMzMzM3QBGZmZmZmRk6QCE7CtejcD3wPxobCZmZmZmZGTpAEQAAAAAAAD1AIUEK16NwPfA/QnkaCSFVVVVVVVWLQBoJIVVVVVVVVYtAGgkhVVVVVVVVi0AaCSFVVVVVVVWLQBoJIVVVVVVVVYtAGgkhVVVVVVVVi0AaCSFVVVVVVVWLQBoJIVVVVVVVVYtAGgkhVVVVVVVVi0AaEhEAAAAAAAA9QCEAAAAAAOBjQCABQg8KDWNudF9hZF9yZXdhcmQaiwYa7gUKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+EW6DO2ukZpQ/GYFRElJEZNA/IJQ+OQAAAAAAACJAQpkCGhIRzczMzMzM7D8heZmZmfkav0AaGwnNzMzMzMzsPxHNzMzMzMz8PyGZmZmZmTk/QBobCc3MzMzMzPw/EZqZmZmZmQVAIWZmZmZmZjlAGhsJmpmZmZmZBUARzczMzMzMDEAhMjMzMzMzE0AaGwnNzMzMzMwMQBEAAAAAAAASQCHMzMzMzMwAQBobCQAAAAAAABJAEZqZmZmZmRVAIdDMzMzMzOw/GhsJmpmZmZmZFUARMzMzMzMzGUAhyMzMzMzM7D8aGwkzMzMzMzMZQBHNzMzMzMwcQCHQzMzMzMzsPxobCc3MzMzMzBxAETMzMzMzMyBAIcjMzMzMzOw/GhsJMzMzMzMzIEARAAAAAAAAIkAh0MzMzMzM7D9CeRoJIQAAAAAAoItAGgkhAAAAAACgi0AaCSEAAAAAAKCLQBoJIQAAAAAAoItAGgkhAAAAAACgi0AaCSEAAAAAAKCLQBoJIQAAAAAAoItAGgkhAAAAAACgi0AaCSEAAAAAAKCLQBoSEQAAAAAAACJAIQAAAAAAwFJAIAFCGAoWY250X2NoYWxsZW5nZV9hX2ZyaWVuZBqeBhqBBgq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4RYkagJF0qwT8ZCqmQpFpY1z8g4TY5AAAAAAAAEEBCmQIaEhGamZmZmZnZPyGZdxIKNGS7QBobCZqZmZmZmdk/EZqZmZmZmek/Ib9+uZNQoAlAGhsJmpmZmZmZ6T8RNDMzMzMz8z8hahybVOhgjkAaGwk0MzMzMzPzPxGamZmZmZn5PyFQuB6F61EIQBobCZqZmZmZmfk/EQAAAAAAAABAIbgehetRuEBAGhsJAAAAAAAAAEARNDMzMzMzA0AhbGZmZmZm9j8aGwk0MzMzMzMDQBFnZmZmZmYGQCFlZmZmZmb2PxobCWdmZmZmZgZAEZqZmZmZmQlAIWVmZmZmZvY/GhsJmpmZmZmZCUARzczMzMzMDEAhZWZmZmZm9j8aGwnNzMzMzMwMQBEAAAAAAAAQQCFlZmZmZmb2P0KLARoJIQAAAAAAYYtAGgkhAAAAAABhi0AaCSEAAAAAAGGLQBoJIQAAAAAAYYtAGgkhAAAAAABhi0AaCSEAAAAAAGGLQBoJIQAAAAAAYYtAGgkhAAAAAABhi0AaEhEAAAAAAADwPyEAAAAAAIiOQBobCQAAAAAAAPA/EQAAAAAAABBAIQAAAAAAgEZAIAFCGAoWY250X2NvbXBsZXRlZF81X2xldmVscxraBhq3Bgq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4RuT9vYhDCAEAZSNBPnrzII0AgsyU5AAAAAACYgkBCmQIaEhEAAAAAAMBNQCFcT+zEzkK/QBobCQAAAAAAwE1AEQAAAAAAwF1AIeVFTBfEhzRAGhsJAAAAAADAXUARAAAAAABQZkAhu9ST8dul7j8aGwkAAAAAAFBmQBEAAAAAAMBtQCG71JPx26XuPxobCQAAAAAAwG1AEQAAAAAAmHJAIbvUk/Hbpe4/GhsJAAAAAACYckARAAAAAABQdkAhu9ST8dul7j8aGwkAAAAAAFB2QBEAAAAAAAh6QCG71JPx26XuPxobCQAAAAAACHpAEQAAAAAAwH1AIbvUk/Hbpe4/GhsJAAAAAADAfUARAAAAAAC8gEAhu9ST8dul7j8aGwkAAAAAALyAQBEAAAAAAJiCQCG71JPx26XuP0LBARoJITMzMzMz641AGgkhMzMzMzPrjUAaCSEzMzMzM+uNQBoJITMzMzMz641AGgkhMzMzMzPrjUAaEhEAAAAAAADwPyEAAAAAAFyDQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAXINAGhsJAAAAAAAA8D8RAAAAAAAAAEAhAAAAAADogkAaGwkAAAAAAAAAQBEAAAAAAAAUQCEAAAAAAAiHQBobCQAAAAAAABRAEQAAAAAAmIJAIQAAAAAAuIRAIAFCHgocY250X2xldmVsX2NvbXBsZXRlX3F1aWNrcGxheRqCBxrkBgq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4R+fTlbjfrE0AZVIbm4kZtMEAgxBcxAAAAAAAA8D85AAAAAAB4hEBCmQIaEhEAAAAAAGBQQCHk////vxW/QBobCQAAAAAAYFBAEQAAAAAAYGBAIezETuzEbkhAGhsJAAAAAABgYEARAAAAAACQaEAh5DEe4zG+MEAaGwkAAAAAAJBoQBEAAAAAAGBwQCF3d3d3d3fxPxobCQAAAAAAYHBAEQAAAAAAeHRAIXd3d3d3d/E/GhsJAAAAAAB4dEARAAAAAACQeEAhd3d3d3d38T8aGwkAAAAAAJB4QBEAAAAAAKh8QCF3d3d3d3fxPxobCQAAAAAAqHxAEQAAAAAAYIBAIXd3d3d3d/E/GhsJAAAAAABggEARAAAAAABsgkAhd3d3d3d38T8aGwkAAAAAAGyCQBEAAAAAAHiEQCF3d3d3d3fxP0LlARoJIQAAAAAAYI9AGgkhAAAAAABgj0AaCSEAAAAAAGCPQBoSEQAAAAAAAPA/IQAAAAAA1INAGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAADUg0AaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAKCLQBobCQAAAAAAAABAEQAAAAAAAAhAIQAAAAAAEIJAGhsJAAAAAAAACEARAAAAAAAAFEAhAAAAAACIhUAaGwkAAAAAAAAUQBEAAAAAAAAmQCEAAAAAAKiKQBobCQAAAAAAACZAEQAAAAAAeIRAIQAAAAAAUIdAIAFCGQoXY250X2xldmVsX2VuZF9xdWlja3BsYXkaswYakwYKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+EfbyBprgxv8/GW7g9qQgnTNAILwxOQAAAAAAiJFAQpkCGhIRzczMzMwMXEAhx4wB0xhPv0AaGwnNzMzMzAxcQBHNzMzMzAxsQCHZI3dzBg8gQBobCc3MzMzMDGxAEZqZmZmZCXVAIez33BKnfu8/GhsJmpmZmZkJdUARzczMzMwMfEAh6vfcEqd+7z8aGwnNzMzMzAx8QBEAAAAAAIiBQCHq99wSp37vPxobCQAAAAAAiIFAEZqZmZmZCYVAIe733BKnfu8/GhsJmpmZmZkJhUARMzMzMzOLiEAh5ffcEqd+7z8aGwkzMzMzM4uIQBHNzMzMzAyMQCHu99wSp37vPxobCc3MzMzMDIxAEWdmZmZmjo9AIe733BKnfu8/GhsJZ2ZmZmaOj0ARAAAAAACIkUAh5ffcEqd+7z9CnQEaCSFJkiRJkkSMQBoJIUmSJEmSRIxAGgkhSZIkSZJEjEAaCSFJkiRJkkSMQBoJIUmSJEmSRIxAGgkhSZIkSZJEjEAaCSFJkiRJkkSMQBoSEQAAAAAAAPA/IQAAAAAAWINAGhsJAAAAAAAA8D8RAAAAAAAAAEAhAAAAAABAckAaGwkAAAAAAAAAQBEAAAAAAIiRQCEAAAAAAKCIQCABQhsKGWNudF9sZXZlbF9yZXNldF9xdWlja3BsYXkalgca9gYKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+EXS8GAg+Wh9AGdPbApEH2jxAIPQPMQAAAAAAAABAOQAAAAAAnJNAQpkCGhIRAAAAAABgX0AheZaWlpYpv0AaGwkAAAAAAGBfQBEAAAAAAGBvQCH+Rtn9RtlDQBobCQAAAAAAYG9AEQAAAAAAiHdAIfrt7/Wl/hlAGhsJAAAAAACId0ARAAAAAABgf0AhI5KGACl+8D8aGwkAAAAAAGB/QBEAAAAAAJyDQCEjkoYAKX7wPxobCQAAAAAAnINAEQAAAAAAiIdAISOShgApfvA/GhsJAAAAAACIh0ARAAAAAAB0i0AhI5KGACl+8D8aGwkAAAAAAHSLQBEAAAAAAGCPQCEjkoYAKX7wPxobCQAAAAAAYI9AEQAAAAAAppFAISOShgApfvA/GhsJAAAAAACmkUARAAAAAACck0AhI5KGACl+8D9C9wEaCSEAAAAAANCPQBoJIQAAAAAA0I9AGhIRAAAAAAAA8D8hAAAAAADIkUAaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAOB/QBobCQAAAAAAAABAEQAAAAAAAABAIQAAAAAA4H9AGhsJAAAAAAAAAEARAAAAAAAACEAhAAAAAAD4hEAaGwkAAAAAAAAIQBEAAAAAAAAUQCEAAAAAAJCLQBobCQAAAAAAABRAEQAAAAAAACBAIQAAAAAAoIVAGhsJAAAAAAAAIEARAAAAAAAAMEAhAAAAAACgiUAaGwkAAAAAAAAwQBEAAAAAAJyTQCEAAAAAACCIQCABQhsKGWNudF9sZXZlbF9zdGFydF9xdWlja3BsYXka+QYa5AYKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+ERBBWfD0hBNAGa996dh/rytAIPcVMQAAAAAAAPA/OQAAAAAAmIJAQpkCGhIRAAAAAADATUAhOAAAAIAtv0AaGwkAAAAAAMBNQBEAAAAAAMBdQCGVNWVNWRNDQBobCQAAAAAAwF1AEQAAAAAAUGZAIcAn8Vx/SxBAGhsJAAAAAABQZkARAAAAAADAbUAhfD0Vafqg8D8aGwkAAAAAAMBtQBEAAAAAAJhyQCF8PRVp+qDwPxobCQAAAAAAmHJAEQAAAAAAUHZAIXw9FWn6oPA/GhsJAAAAAABQdkARAAAAAAAIekAhfD0Vafqg8D8aGwkAAAAAAAh6QBEAAAAAAMB9QCF8PRVp+qDwPxobCQAAAAAAwH1AEQAAAAAAvIBAIXw9FWn6oPA/GhsJAAAAAAC8gEARAAAAAACYgkAhfD0Vafqg8D9C5QEaCSFVVVVVVT2NQBoJIVVVVVVVPY1AGgkhVVVVVVU9jUAaEhEAAAAAAADwPyEAAAAAAIiEQBobCQAAAAAAAPA/EQAAAAAAAPA/IQAAAAAAiIRAGhsJAAAAAAAA8D8RAAAAAAAAAEAhAAAAAABIh0AaGwkAAAAAAAAAQBEAAAAAAAAQQCEAAAAAAFCMQBobCQAAAAAAABBAEQAAAAAAABhAIQAAAAAAEIZAGhsJAAAAAAAAGEARAAAAAAAAKEAhAAAAAADoh0AaGwkAAAAAAAAoQBEAAAAAAJiCQCEAAAAAAKCIQCABQhAKDmNudF9wb3N0X3Njb3JlGqIGGoEGCrYCCN8+GAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAgAUDfPhExTnVgkvDbPxmQfyrht8T8PyCRNjkAAAAAAABKQEKZAhoSEc3MzMzMzBRAIREzMzNz1L5AGhsJzczMzMzMFEARzczMzMzMJEAhMzMzMzMjWEAaGwnNzMzMzMwkQBE0MzMzMzMvQCEzMzMzMzMwQBobCTQzMzMzMy9AEc3MzMzMzDRAIZmZmZmZ+TJAGhsJzczMzMzMNEARAAAAAAAAOkAhMjMzMzMz8j8aGwkAAAAAAAA6QBE0MzMzMzM/QCE2MzMzMzPyPxobCTQzMzMzMz9AETMzMzMzM0JAIS8zMzMzM/I/GhsJMzMzMzMzQkARzczMzMzMREAhNjMzMzMz8j8aGwnNzMzMzMxEQBFnZmZmZmZHQCE2MzMzMzPyPxobCWdmZmZmZkdAEQAAAAAAAEpAIS8zMzMzM/I/QosBGgkhAAAAAAARi0AaCSEAAAAAABGLQBoJIQAAAAAAEYtAGgkhAAAAAAARi0AaCSEAAAAAABGLQBoJIQAAAAAAEYtAGgkhAAAAAAARi0AaCSEAAAAAABGLQBoSEQAAAAAAAPA/IQAAAAAAAHxAGhsJAAAAAAAA8D8RAAAAAAAASkAhAAAAAABwhEAgAUIcChpjbnRfc3BlbmRfdmlydHVhbF9jdXJyZW5jeRqbBhqBBgq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4RwUfrtWtT2T8Za3ms0V0X+z8ggjc5AAAAAAAASkBCmQIaEhHNzMzMzMwUQCE8zczMjNy+QBobCc3MzMzMzBRAEc3MzMzMzCRAIc3MzMzM3FdAGhsJzczMzMzMJEARNDMzMzMzL0AhXI/C9ShcL0AaGwk0MzMzMzMvQBHNzMzMzMw0QCFvPQrXo/AqQBobCc3MzMzMzDRAEQAAAAAAADpAITIzMzMzM+8/GhsJAAAAAAAAOkARNDMzMzMzP0AhODMzMzMz7z8aGwk0MzMzMzM/QBEzMzMzMzNCQCEsMzMzMzPvPxobCTMzMzMzM0JAEc3MzMzMzERAITgzMzMzM+8/GhsJzczMzMzMREARZ2ZmZmZmR0AhODMzMzMz7z8aGwlnZmZmZmZHQBEAAAAAAABKQCEsMzMzMzPvP0KLARoJIQAAAAAAgotAGgkhAAAAAACCi0AaCSEAAAAAAIKLQBoJIQAAAAAAgotAGgkhAAAAAACCi0AaCSEAAAAAAIKLQBoJIQAAAAAAgotAGgkhAAAAAACCi0AaEhEAAAAAAADwPyEAAAAAAFB4QBobCQAAAAAAAPA/EQAAAAAAAEpAIQAAAAAAwIJAIAFCFQoTY250X3VzZV9leHRyYV9zdGVwcxrMBxqyBwq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4R9ijl9eLQPkAZQmxuMDmcSkApAAAAAAAA8D8xAAAAAAAALEA5AAAAAAAAjUBCogIaGwkAAAAAAADwPxHNzMzMzGxXQCFlZmZm5kK9QBobCc3MzMzMbFdAEc3MzMzMTGdAIQAAAAAAGHdAGhsJzczMzMxMZ0ARmpmZmZlxcUAhoQ7qoA6qWEAaGwmamZmZmXFxQBHNzMzMzDx3QCGCiw0uNrhFQBobCc3MzMzMPHdAEQAAAAAACH1AIRyQrHPlgShAGhsJAAAAAAAIfUARmpmZmZlpgUAhkSzMdY/+IUAaGwmamZmZmWmBQBEzMzMzM0+EQCHZpmcU4QL9PxobCTMzMzMzT4RAEc3MzMzMNIdAIeOmZxThAv0/GhsJzczMzMw0h0ARZ2ZmZmYaikAh46ZnFOEC/T8aGwlnZmZmZhqKQBEAAAAAAACNQCHZpmcU4QL9P0KkAhobCQAAAAAAAPA/EQAAAAAAAABAIQAAAAAASIlAGhsJAAAAAAAAAEARAAAAAAAAEEAhAAAAAACAiUAaGwkAAAAAAAAQQBEAAAAAAAAcQCEAAAAAAPiNQBobCQAAAAAAABxAEQAAAAAAACRAIQAAAAAAQIdAGhsJAAAAAAAAJEARAAAAAAAALEAhAAAAAABIiUAaGwkAAAAAAAAsQBEAAAAAAAA0QCEAAAAAACiIQBobCQAAAAAAADRAEQAAAAAAADxAIQAAAAAA6IZAGhsJAAAAAAAAPEARAAAAAACAREAhAAAAAADIiEAaGwkAAAAAAIBEQBEAAAAAAMBRQCEAAAAAAOiIQBobCQAAAAAAwFFAEQAAAAAAAI1AIQAAAAAA8IhAIAFCFQoTY250X3VzZXJfZW5nYWdlbWVudBrqFBACItoUCrYCCN8+GAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAgAUDfPhB/GhgSDVVuaXRlZCBTdGF0ZXMZAAAAAAArsUAaEBIFSW5kaWEZAAAAAABgfkAaEBIFSmFwYW4ZAAAAAAAgfEAaERIGQ2FuYWRhGQAAAAAAIHZAGhQSCUF1c3RyYWxpYRkAAAAAAHB0QBoZEg5Vbml0ZWQgS2luZ2RvbRkAAAAAAPByQBoSEgdHZXJtYW55GQAAAAAAAGJAGhESBk1leGljbxkAAAAAAIBZQBoREgZGcmFuY2UZAAAAAABAWEAaERIGQnJhemlsGQAAAAAAQFdAGhESBlRhaXdhbhkAAAAAAABSQBoQEgVDaGluYRkAAAAAAEBQQBoXEgxTYXVkaSBBcmFiaWEZAAAAAACASEAaExIIUGFraXN0YW4ZAAAAAAAASEAaEBIFRWd5cHQZAAAAAAAAR0AaFhILTmV0aGVybGFuZHMZAAAAAACARkAaEhIHVmlldG5hbRkAAAAAAABFQBoWEgtQaGlsaXBwaW5lcxkAAAAAAIBDQBoXEgxTb3V0aCBBZnJpY2EZAAAAAAAAQ0AaERIGSXNyYWVsGQAAAAAAgEJAGhQSCUluZG9uZXNpYRkAAAAAAABBQBoQEgVTcGFpbhkAAAAAAAA/QBofEhRVbml0ZWQgQXJhYiBFbWlyYXRlcxkAAAAAAAA9QBoUEglBcmdlbnRpbmEZAAAAAAAAPUAaFhILTmV3IFplYWxhbmQZAAAAAAAAPEAaEBIFSXRhbHkZAAAAAAAAPEAaExIIVGhhaWxhbmQZAAAAAAAAOUAaEhIHTGViYW5vbhkAAAAAAAA5QBoTEghDb2xvbWJpYRkAAAAAAAA4QBoVEgpCYW5nbGFkZXNoGQAAAAAAADhAGhQSCVNpbmdhcG9yZRkAAAAAAAA1QBoTEghNYWxheXNpYRkAAAAAAAA1QBoREgZTd2VkZW4ZAAAAAAAAMkAaFBIJSG9uZyBLb25nGQAAAAAAADJAGhYSC1N3aXR6ZXJsYW5kGQAAAAAAADFAGhESBlJ1c3NpYRkAAAAAAAAxQBoREgZQb2xhbmQZAAAAAAAALkAaEhIHQXVzdHJpYRkAAAAAAAAuQBoSEgdEZW5tYXJrGQAAAAAAACxAGhASBUNoaWxlGQAAAAAAACxAGhMSCFBvcnR1Z2FsGQAAAAAAACpAGhESBkpvcmRhbhkAAAAAAAAqQBoREgZUdXJrZXkZAAAAAAAAKEAaDxIEUGVydRkAAAAAAAAoQBoWEgtTb3V0aCBLb3JlYRkAAAAAAAAmQBoREgZOb3J3YXkZAAAAAAAAJkAaExIIU2xvdmVuaWEZAAAAAAAAJEAaEBIFTmVwYWwZAAAAAAAAJEAaEhIHTW9yb2NjbxkAAAAAAAAkQBoREgZHcmVlY2UZAAAAAAAAJEAaEhIHQ3plY2hpYRkAAAAAAAAkQBoSEgdCZWxnaXVtGQAAAAAAACRAGhYSC1B1ZXJ0byBSaWNvGQAAAAAAACBAGhISB0VjdWFkb3IZAAAAAAAAIEAaERIGU2VyYmlhGQAAAAAAABxAGhISB05pZ2VyaWEZAAAAAAAAHEAaEhIHSXJlbGFuZBkAAAAAAAAcQBoSEgdIdW5nYXJ5GQAAAAAAABxAGhISB1VrcmFpbmUZAAAAAAAAGEAaEhIHVHVuaXNpYRkAAAAAAAAYQBocEhFUcmluaWRhZCAmIFRvYmFnbxkAAAAAAAAYQBoSEgdSb21hbmlhGQAAAAAAABhAGh0SEkRvbWluaWNhbiBSZXB1YmxpYxkAAAAAAAAYQBoSEgdCYWhyYWluGQAAAAAAABhAGhISB0FsZ2VyaWEZAAAAAAAAGEAaFBIJU3JpIExhbmthGQAAAAAAABRAGhASBVFhdGFyGQAAAAAAABRAGhASBUtlbnlhGQAAAAAAABRAGhUSCkNvc3RhIFJpY2EZAAAAAAAAFEAaExIIVGFuemFuaWEZAAAAAAAAEEAaDxIESXJhbhkAAAAAAAAQQBoTEghCdWxnYXJpYRkAAAAAAAAQQBoSEgdCYWhhbWFzGQAAAAAAABBAGhMSCFppbWJhYndlGQAAAAAAAAhAGhMSCFNsb3Zha2lhGQAAAAAAAAhAGhESBlBhbmFtYRkAAAAAAAAIQBoaEg9NeWFubWFyIChCdXJtYSkZAAAAAAAACEAaEBIFTWFjYXUZAAAAAAAACEAaFBIJTGl0aHVhbmlhGQAAAAAAAAhAGhESBkt1d2FpdBkAAAAAAAAIQBoSEgdKYW1haWNhGQAAAAAAAAhAGg8SBElyYXEZAAAAAAAACEAaEhIHRmlubGFuZBkAAAAAAAAIQBoWEgtFbCBTYWx2YWRvchkAAAAAAAAIQBoSEgdDcm9hdGlhGQAAAAAAAAhAGhISB0JvbGl2aWEZAAAAAAAACEAaCRkAAAAAAAAIQBoREgZaYW1iaWEZAAAAAAAAAEAaFBIJVmVuZXp1ZWxhGQAAAAAAAABAGhASBVN5cmlhGQAAAAAAAABAGhsSEFBhcHVhIE5ldyBHdWluZWEZAAAAAAAAAEAaExIITW9uZ29saWEZAAAAAAAAAEAaFBIJTWF1cml0aXVzGQAAAAAAAABAGhESBkxhdHZpYRkAAAAAAAAAQBoVEgpLYXpha2hzdGFuGQAAAAAAAABAGhMSCEhvbmR1cmFzGQAAAAAAAABAGg8SBEd1YW0ZAAAAAAAAAEAaEBIFR2hhbmEZAAAAAAAAAEAaGxIQQ8O0dGUgZOKAmUl2b2lyZRkAAAAAAAAAQBoREgZDeXBydXMZAAAAAAAAAEAlEN0nQSrzAQoYIg1Vbml0ZWQgU3RhdGVzKQAAAAAAK7FAChQIARABIgVJbmRpYSkAAAAAAGB+QAoUCAIQAiIFSmFwYW4pAAAAAAAgfEAKFQgDEAMiBkNhbmFkYSkAAAAAACB2QAoYCAQQBCIJQXVzdHJhbGlhKQAAAAAAcHRACh0IBRAFIg5Vbml0ZWQgS2luZ2RvbSkAAAAAAPByQAoWCAYQBiIHR2VybWFueSkAAAAAAABiQAoVCAcQByIGTWV4aWNvKQAAAAAAgFlAChUICBAIIgZGcmFuY2UpAAAAAABAWEAKFQgJEAkiBkJyYXppbCkAAAAAAEBXQEIJCgdjb3VudHJ5GsIHGrIHCrYCCN8+GAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAgAUDfPhErSNjhjYMPQBl6KOCzl0b/PykAAAAAAADwPzEAAAAAAAAQQDkAAAAAAAAcQEKiAhobCQAAAAAAAPA/EZqZmZmZmfk/IdYSp35PU5BAGhsJmpmZmZmZ+T8RmpmZmZmZAUAhkFO/5xYPk0AaGwmamZmZmZkBQBFmZmZmZmYGQCEwMzMzMzMTQBobCWZmZmZmZgZAETMzMzMzMwtAIWGg/ZxLE5VAGhsJMzMzMzMzC0ARAAAAAAAAEEAh/cVoyRr3kkAaGwkAAAAAAAAQQBFmZmZmZmYSQCHWi+fftU4TQBobCWZmZmZmZhJAEc3MzMzMzBRAIbWD1JAcy5BAGhsJzczMzMzMFEARMzMzMzMzF0AhGtRBHdRBE0AaGwkzMzMzMzMXQBGZmZmZmZkZQCF7HXqaM36PQBobCZmZmZmZmRlAEQAAAAAAABxAIS1E8Rc5S5FAQqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hAAAAAABAkEAaGwkAAAAAAADwPxEAAAAAAAAAQCEAAAAAAByTQBobCQAAAAAAAABAEQAAAAAAAAhAIQAAAAAAIIVAGhsJAAAAAAAACEARAAAAAAAACEAhAAAAAAAghUAaGwkAAAAAAAAIQBEAAAAAAAAQQCEAAAAAAASDQBobCQAAAAAAABBAEQAAAAAAABBAIQAAAAAABINAGhsJAAAAAAAAEEARAAAAAAAAFEAhAAAAAADYkEAaGwkAAAAAAAAUQBEAAAAAAAAYQCEAAAAAAJiPQBobCQAAAAAAABhAEQAAAAAAABxAIQAAAAAAWIFAGhsJAAAAAAAAHEARAAAAAAAAHEAhAAAAAABYgUAgAUILCglkYXlvZndlZWsawgcasgcKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+Eedg4B/okmlAGajINRCoW0FAKQAAAAAAYGRAMQAAAAAAQGhAOQAAAAAAUHFAQqICGhsJAAAAAABgZEARzczMzMzMZUAhYGZmZmY8oEAaGwnNzMzMzMxlQBGamZmZmTlnQCGiqqqqqnKWQBobCZqZmZmZOWdAEWZmZmZmpmhAIZuZmZmZ9YVAGhsJZmZmZmamaEARMzMzMzMTakAhERERERHggEAaGwkzMzMzMxNqQBEAAAAAAIBrQCFnZmZmZuODQBobCQAAAAAAgGtAEc3MzMzM7GxAIVK4HoXrcXxAGhsJzczMzMzsbEARmpmZmZlZbkAhFq5H4XrEdkAaGwmamZmZmVluQBFmZmZmZsZvQCGRmZmZmaGLQBobCWZmZmZmxm9AEZqZmZmZmXBAIZyZmZmZeYBAGhsJmpmZmZmZcEARAAAAAABQcUAhMTMzMzNjeUBCpAIaGwkAAAAAAGBkQBEAAAAAAMBkQCEAAAAAALiKQBobCQAAAAAAwGRAEQAAAAAAYGVAIQAAAAAAkIlAGhsJAAAAAABgZUARAAAAAAAAZkAhAAAAAABwh0AaGwkAAAAAAABmQBEAAAAAAMBmQCEAAAAAAMiKQBobCQAAAAAAwGZAEQAAAAAAQGhAIQAAAAAAmIhAGhsJAAAAAABAaEARAAAAAABAakAhAAAAAAAIiUAaGwkAAAAAAEBqQBEAAAAAAEBsQCEAAAAAAJiIQBobCQAAAAAAQGxAEQAAAAAA4G5AIQAAAAAA+IpAGhsJAAAAAADgbkARAAAAAAAQcEAhAAAAAADwhkAaGwkAAAAAABBwQBEAAAAAAFBxQCEAAAAAAFiIQCABQgsKCWp1bGlhbmRheRruAxACIuADCrYCCN8+GAEgAS0AAIA/MqQCGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAgAUDfPhAFGgwSATYZAAAAAABqqEAaDBIBNxkAAAAAALicQBoMEgE5GQAAAAAA2JpAGgwSATgZAAAAAADwk0AaDRICMTAZAAAAAACAUkAl7y2BPypXCgwiATYpAAAAAABqqEAKEAgBEAEiATcpAAAAAAC4nEAKEAgCEAIiATkpAAAAAADYmkAKEAgDEAMiATgpAAAAAADwk0AKEQgEEAQiAjEwKQAAAAAAgFJAQgcKBW1vbnRoGsEDEAIiqAMKtgII3z4YASABLQAAgD8ypAIaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQCABQN8+EAMaDhIDSU9TGQAAAAAAGK9AGhISB0FORFJPSUQZAAAAAACsrUAaCRkAAAAAAKBvQCVYgplAKjcKDiIDSU9TKQAAAAAAGK9AChYIARABIgdBTkRST0lEKQAAAAAArK1ACg0IAhACKQAAAAAAoG9AQhIKEG9wZXJhdGluZ19zeXN0ZW0a1SkQAiK+KQq2AgjfPhgBIAEtAACAPzKkAhobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAGhsJAAAAAAAA8D8RAAAAAAAA8D8hzczMzMwYiUAaGwkAAAAAAADwPxEAAAAAAADwPyHNzMzMzBiJQBobCQAAAAAAAPA/EQAAAAAAAPA/Ic3MzMzMGIlAIAFA3z4Q3z4aKxIgRkZGNzBGMjFEQTc0NjdCNzE3RUUyMzEzRkFFNzVBNzgZAAAAAAAAGEAaKxIgRkZEREYzNTREOUI5RTg1MUQyQTYyQUJBRUExN0RBM0UZAAAAAAAAGEAaKxIgRkY1MkE2MkY2QTBBOEQ4NUU4NTM1NzJGRUUwOUUxNTAZAAAAAAAAGEAaKxIgRkYyNTc0OTVERjBENjBBNURCQzQ2QUFFNkY3ODMwOEIZAAAAAAAAGEAaKxIgRkYyMEU0RUUxMjlDMzcwMEZDMUM2Q0VGMTg4RTE3QkIZAAAAAAAAGEAaKxIgRkU4NUYwMDk3NEE0MjkzMENBMEExREFCNDQ2QjNGRjEZAAAAAAAAGEAaKxIgRkU1NzE4MTM2MEFEQ0Q4NkI3RTA5MDYzREYyQkJEMkUZAAAAAAAAGEAaKxIgRkU0QTIyMDA2MURENEY5QTI1MTY3QUE1MTlDRDA2MjAZAAAAAAAAGEAaKxIgRkU0MTkxREZFRUYzMEZERjYwMTlBODNGMkJDQzhGODQZAAAAAAAAGEAaKxIgRkUxNDA2RTE0OUE2MDg0RERGODU5Q0U5QUQ0MzE1NzkZAAAAAAAAGEAaKxIgRkUwRDI1NzgxNEFBNzJBNEVBMzVFODZBNTkyOTdGMzkZAAAAAAAAGEAaKxIgRkRDRTc5N0UyNzQzMzUzMDc5NjVGRkNEQkNEQkMxOTEZAAAAAAAAGEAaKxIgRkQ3NTVDNkUxRjM4QTY3NkRFRTdGMjA5QTUxNkNFQjUZAAAAAAAAGEAaKxIgRkQ0NkM5QkFDMEVFNjRBNDZDQzQ2ODdEQzY4MjAxQ0IZAAAAAAAAGEAaKxIgRkNGRjBEN0MxOUE5MDJDNERCNEVBRjhEQzU4NjEwNUIZAAAAAAAAGEAaKxIgRkM2OEZENEMxOTQ5REM4Qjk2Rjk1OTI4MTEyOEZCQzgZAAAAAAAAGEAaKxIgRkM1RTgzQkYyNTdCNjVDMDA2REM0MkNFQjQ1NUQ3MzYZAAAAAAAAGEAaKxIgRkI5MzExMTg3MEU4ODIzMkNDMTBFQTdFQTJCQzEyNzIZAAAAAAAAGEAaKxIgRkI0MDc3NTNCMDRERjQ1QjFFODlCRjBBNjc4RUY5MkUZAAAAAAAAGEAaKxIgRkIwNkFGMUJEM0YzNzM3RDFGRjc3QTk0OTgyM0E1ODAZAAAAAAAAGEAaKxIgRkFGOEIyRDYwN0NFOTdBQTkwOTgyRTU5RTY3MzRFMDYZAAAAAAAAGEAaKxIgRjlGRUY1QkMyNUE4QzIwMzRFMzE3ODQ3QTY0NTJEQzgZAAAAAAAAGEAaKxIgRjlCMUZFQTc1Qjg5NkRBNzI3QkM3MzMxNzNCMTlENjAZAAAAAAAAGEAaKxIgRjk5Q0MzNTk4NTIyQUJFNTRCOURDQUMyOTY5MTc5Q0MZAAAAAAAAGEAaKxIgRjk2QzU0QTU2RTAwMDIxMDNDNkQ5MEQ0MkM1N0E5QUQZAAAAAAAAGEAaKxIgRjkxNkQzMEZBRDM2RjEyODJGOThBRUEzRDU4MEJFQkUZAAAAAAAAGEAaKxIgRjhFQkQ4MzE2QzBGQUE5MDZEMTMzNUIzRDJDMDRDODMZAAAAAAAAGEAaKxIgRjhDRkExNzcxNzJGNjlDRDNCNjkxREI2MjFFODBDRDIZAAAAAAAAGEAaKxIgRjgxRTA0MzBEODQ3OEQ2N0FFMDFGNzNEOTg2NTZDQ0EZAAAAAAAAGEAaKxIgRjgxQTdGQUNEOTY0MEY4MTY5RDhERTJFMjdEMjE3RTYZAAAAAAAAGEAaKxIgRjczMTU5NDhEMkY4QTdGM0QwQkYzNzE4N0QzQkE4QUYZAAAAAAAAGEAaKxIgRjczMTA5MDNEQkQ1NUM4QjE0M0YwRDAxRUVBQ0YyQzkZAAAAAAAAGEAaKxIgRjZBMEVEQzk1RDM5MjBCRkNFNDE4Q0U3QUNDM0M4RkYZAAAAAAAAGEAaKxIgRjY3RUE4NURCRUY3NTZBMTQ3NTg0NTMzNERBODNBM0UZAAAAAAAAGEAaKxIgRjY3NTU4ODdEMUY3NzFFRTAxOEM4RjQ5NTFCN0FFMTEZAAAAAAAAGEAaKxIgRjY2QjAxQTZBQjM0OTI3RjMwRkZCMjY5MTNDRTA4MUUZAAAAAAAAGEAaKxIgRjY0NTJDRDIyMUM0RkRFMEFGNTlDMkREMzc5QUMyOEUZAAAAAAAAGEAaKxIgRjYyM0QzOUQzRjM3MDNDNTdFNTIwNDI5MUZBODQ1NDcZAAAAAAAAGEAaKxIgRjVCM0M1MjBCNkMxNkFFRjIwMUE2MDM5M0RENUFFOTEZAAAAAAAAGEAaKxIgRjVBQTEyQjFFMTlBRDE2NjU1MzdBNkJCRTNGRDY3NkIZAAAAAAAAGEAaKxIgRjUyMzRFMjg2RUVEQUI3ODY0RTY4MEFBODc0QTJEQzUZAAAAAAAAGEAaKxIgRjRDQjBCMzBFMEExNzBBRUI2QTk0RUYxMTU2NzM4QzgZAAAAAAAAGEAaKxIgRjQ4OEI0NTM2QkExMDRGRTM5RUQ2M0YyRDQ5NTlCOUMZAAAAAAAAGEAaKxIgRjQ4MTBCMzVEMDg0MTA1NDI3NDExMzdCRDVGRDMzMjYZAAAAAAAAGEAaKxIgRjQ4MDY4RDQ4RjEyODM5NkQxMDA2ODY1NEJEMUNCNzQZAAAAAAAAGEAaKxIgRjQ3NkM2NDlCQjEyN0ZBMTNFMEU5MUYzRDY2MkYwQjYZAAAAAAAAGEAaKxIgRjQ3NUEyM0VFMEI4MERDNTkyNTBDQkIxQzM1QUZCM0UZAAAAAAAAGEAaKxIgRjNDOUQ3MjM1QzJBNjBGQkZCREUxRjJFRUZCQThCMkIZAAAAAAAAGEAaKxIgRjNCODI4RjY0OTRCNzk3MkVCNTIxQzUyODAxMkMyQzIZAAAAAAAAGEAaKxIgRjM2MDQ0Njc2NjM2RDE0Rjg2RkQwODNEREQ5MTBFMDQZAAAAAAAAGEAaKxIgRjM0RkYxQzlGMEMwMjJCNjg0QkM4ODVCQTFEREMyNjAZAAAAAAAAGEAaKxIgRjMyQTE5QjQ0MEQ0ODc2NTkzQkQ1ODkwRDk4MjE1NzIZAAAAAAAAGEAaKxIgRjMwQUFBNTE0MzY5NDY4MjAwMEM1NTMxMjJBRDE1QUIZAAAAAAAAGEAaKxIgRjJENEMxMTM1MDA2NDAzMUVBOEFDNTBBQTYzRjk0QTgZAAAAAAAAGEAaKxIgRjJEMEI3RTg2NjdFNDg1NUJFRkQzOUVDQTMxODVCRjcZAAAAAAAAGEAaKxIgRjJBMUZEOEIxNDJFQjRDNkNBQUU0MTVGQTE5MTdGNTAZAAAAAAAAGEAaKxIgRjI3RjA0NkRGOTYwQUI1MENFNjVCOUY0MzY0Qzk4MUUZAAAAAAAAGEAaKxIgRjI0QkJEMzYyMDgwNDJDNTg3Q0VFRkJGNjk1MUFBMkIZAAAAAAAAGEAaKxIgRjIwOTJFQzMxODc5NjIwNTlBQ0MxQ0E1NzBCQUZEQTcZAAAAAAAAGEAaKxIgRjFGQzdBMUU2OTg2MzM2RDI1Qjg5QjdFMDExMkU3NjAZAAAAAAAAGEAaKxIgRjFCODQxQjY0NUU5NjI0OUJERkQ1OURCNzY4NjUyMTEZAAAAAAAAGEAaKxIgRjE3RUE2QTNCNTc5NjNGNDRGN0I4MUM1RDc2MUY5RjIZAAAAAAAAGEAaKxIgRjEwRTNBQ0NGRUE4NEQyNTI5N0NGNkJDNzU4MDNBRUYZAAAAAAAAGEAaKxIgRjBGQzk5RjU2M0UxMjQ3QUQ3MDI0MEU1QUExMzJBQkYZAAAAAAAAGEAaKxIgRjBENjlDMUEyRUREMjEwNjgxNjJCM0UxOEQwN0EwMzAZAAAAAAAAGEAaKxIgRjBBNjdCNDE1NEY0Mjc1QjMyMjQzMjI4RUM2MUJENzQZAAAAAAAAGEAaKxIgRjA1RTdERDcwNjBCQjk4MUE2MEI2OTA1NDg3MEUzOUIZAAAAAAAAGEAaKxIgRjA0RDVGRTczRjNGMDQ0NDM4MEJGRUIxMjNGNzFEMDUZAAAAAAAAGEAaKxIgRUY5RThDQ0NDQkJCMDE1Q0FFMEMzMzRBQjE3NDU0MTcZAAAAAAAAGEAaKxIgRUY4REQzRDc4MzgwMUY2NERBNDFDQzQ4OTJBOTgwMUIZAAAAAAAAGEAaKxIgRUY4NUJCMUQxOTk5NkZERTcwNjEwMDFDMkIzQ0ZCREYZAAAAAAAAGEAaKxIgRUY3MDIxQTI1QjA0NEQ3NkI3QTlERkQwNzA3NDY2MkMZAAAAAAAAGEAaKxIgRUVCMEREOEI2NkE5NDhDRkI3QTJFQ0ExQ0U4MjNGNkUZAAAAAAAAGEAaKxIgRUUyMDEwRkZEQUY2NjNCQ0M3MjIxNjBBQjk5NTRBMUUZAAAAAAAAGEAaKxIgRUUwN0Y1MkE2MThCQTdBQUNEQ0Q2NzEyQjAxRjk0NUEZAAAAAAAAGEAaKxIgRUREN0NEMTQ0Q0YyMjdGQjU0NDUzMzkwRjNDMDY0MDgZAAAAAAAAGEAaKxIgRURENjYxMzBBMENGRERGMzk3RjNCQURGOUY3NkVFQkYZAAAAAAAAGEAaKxIgRURDMEZDM0FFNjhCRjA0MTA5NjIyQUJCREI4RjAxNzEZAAAAAAAAGEAaKxIgRUNDNjk3NUVCOTgwOEZCQjQ1MzkwRTNFREI3NUIyQTEZAAAAAAAAGEAaKxIgRUM4OTVDNzMyMDkyQkE5QTNDRTlGNTA3RUQ4ODg3MjYZAAAAAAAAGEAaKxIgRUM3RjU0OTYwRjc4OUVCMTMwMjY3RjlBM0YyNDdCQjEZAAAAAAAAGEAaKxIgRUM2QjgxRjI4OEY5ODVGNjc5Q0MxODMwNTFFODRCNzIZAAAAAAAAGEAaKxIgRUJGRTA2MjNFRjExMDlBODlCMzkyQjlERjQzNzU5NTEZAAAAAAAAGEAaKxIgRUJDOTNDNEEzNjRGNjgzMEZFQ0IxNkM2NDgyRjBENTgZAAAAAAAAGEAaKxIgRUIyREZGNjYzMjI3NUNEMkM3MjUxQTA0NTA0MTk5MjYZAAAAAAAAGEAaKxIgRUIxQTQzMEJBQ0IzN0U2RUE3MjU5MkJGODM4RTY0QTYZAAAAAAAAGEAaKxIgRUIxMzY4MUMwMUUxQjdERjhGRDBENEVFMjlBMTU1RjIZAAAAAAAAGEAaKxIgRUFGRDZGMjYwNEM4MzUxMDMyNzY0RTVGOEI0RDM4OUUZAAAAAAAAGEAaKxIgRUFFRDkxMERCMDgyQTkxNjQ1RTg5RjdCMDk3MEIxMjkZAAAAAAAAGEAaKxIgRUFFMkQ4NEIwMDhDREI0RkIyMEQ5QzdGNEJFNDIzMUUZAAAAAAAAGEAaKxIgRUE5QTE2RDkwNDIyODZENTUzOTlBNDE4Q0M3QTI4RjMZAAAAAAAAGEAaKxIgRUE5NTEwNEU1OTdFNkE4MUJEQjJGNUE3N0RBRTU5NTEZAAAAAAAAGEAaKxIgRUE4OEUxRURGQzBGNzZDQTE3M0U5MUM3NzNGRkM4RkIZAAAAAAAAGEAaKxIgRUE2MjE5NkMzQ0QzQTE2ODc2MUVERjg1MUM0NEQyOTMZAAAAAAAAGEAaKxIgRTlCNUVDNzhGMUY0RDZGOTBCNUFGQjM0RDA5Njk3RjIZAAAAAAAAGEAaKxIgRTk3Q0RFNUVCMURGMDMwNTBBNUIxMTgxQjg3NjlBQkUZAAAAAAAAGEAaKxIgRTkzMjgxQTI1NkZDMzdEREE1MDJCNjRCNkUyMTAyODIZAAAAAAAAGEAaKxIgRTkzMTkyQ0QyRUMzRDE1OUU5QTVCMTA1RTQ3OUU3OEQZAAAAAAAAGEAaKxIgRTkyMEQ3MkFGQjE2MzY1Nzk3OTU1RTQ2OTEzRTI1OTkZAAAAAAAAGEAaKxIgRTkxQjg2NjdCQzlBN0VCMjI5OUIyRUFGQkNGMUI5RTcZAAAAAAAAGEAlAAAAQirmAworIiBGRkY3MEYyMURBNzQ2N0I3MTdFRTIzMTNGQUU3NUE3OCkAAAAAAAAYQAovCAEQASIgRkZEREYzNTREOUI5RTg1MUQyQTYyQUJBRUExN0RBM0UpAAAAAAAAGEAKLwgCEAIiIEZGNTJBNjJGNkEwQThEODVFODUzNTcyRkVFMDlFMTUwKQAAAAAAABhACi8IAxADIiBGRjI1NzQ5NURGMEQ2MEE1REJDNDZBQUU2Rjc4MzA4QikAAAAAAAAYQAovCAQQBCIgRkYyMEU0RUUxMjlDMzcwMEZDMUM2Q0VGMTg4RTE3QkIpAAAAAAAAGEAKLwgFEAUiIEZFODVGMDA5NzRBNDI5MzBDQTBBMURBQjQ0NkIzRkYxKQAAAAAAABhACi8IBhAGIiBGRTU3MTgxMzYwQURDRDg2QjdFMDkwNjNERjJCQkQyRSkAAAAAAAAYQAovCAcQByIgRkU0QTIyMDA2MURENEY5QTI1MTY3QUE1MTlDRDA2MjApAAAAAAAAGEAKLwgIEAgiIEZFNDE5MURGRUVGMzBGREY2MDE5QTgzRjJCQ0M4Rjg0KQAAAAAAABhACi8ICRAJIiBGRTE0MDZFMTQ5QTYwODREREY4NTlDRTlBRDQzMTU3OSkAAAAAAAAYQEIQCg51c2VyX3BzZXVkb19pZA==\"></facets-overview>';\n",
       "        facets_iframe.srcdoc = facets_html;\n",
       "         facets_iframe.id = \"\";\n",
       "         setTimeout(() => {\n",
       "           facets_iframe.setAttribute('height', facets_iframe.contentWindow.document.body.offsetHeight + 'px')\n",
       "         }, 1500)\n",
       "         </script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Anomaly short description</th>\n",
       "      <th>Anomaly long description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Feature name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'cnt_completed_5_levels'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.549339 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_level_reset_quickplay'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.498636 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_user_engagement'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.0929705 (up to six significant digits), above the threshold 0.004.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_level_end_quickplay'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.343587 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_ad_reward'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.672075 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_use_extra_steps'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.390741 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_post_score'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.352555 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_level_complete_quickplay'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.48113 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'country'</th>\n",
       "      <td>High Linfty distance between current and previous</td>\n",
       "      <td>The Linfty distance between current and previous is 0.0168421 (up to six significant digits), above the threshold 0.003. The feature value with maximum difference is: Taiwan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_spend_virtual_currency'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.332597 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_challenge_a_friend'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.737841 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>'cnt_level_start_quickplay'</th>\n",
       "      <td>High approximate Jensen-Shannon divergence between current and previous</td>\n",
       "      <td>The approximate Jensen-Shannon divergence between current and previous is 0.363518 (up to six significant digits), above the threshold 0.3.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                              Anomaly short description  \\\n",
       "Feature name                                                                                              \n",
       "'cnt_completed_5_levels'        High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_level_reset_quickplay'     High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_user_engagement'           High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_level_end_quickplay'       High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_ad_reward'                 High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_use_extra_steps'           High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_post_score'                High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_level_complete_quickplay'  High approximate Jensen-Shannon divergence between current and previous   \n",
       "'country'                                             High Linfty distance between current and previous   \n",
       "'cnt_spend_virtual_currency'    High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_challenge_a_friend'        High approximate Jensen-Shannon divergence between current and previous   \n",
       "'cnt_level_start_quickplay'     High approximate Jensen-Shannon divergence between current and previous   \n",
       "\n",
       "                                                                                                                                                                                     Anomaly long description  \n",
       "Feature name                                                                                                                                                                                                   \n",
       "'cnt_completed_5_levels'                                          The approximate Jensen-Shannon divergence between current and previous is 0.549339 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_level_reset_quickplay'                                       The approximate Jensen-Shannon divergence between current and previous is 0.498636 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_user_engagement'                                          The approximate Jensen-Shannon divergence between current and previous is 0.0929705 (up to six significant digits), above the threshold 0.004.  \n",
       "'cnt_level_end_quickplay'                                         The approximate Jensen-Shannon divergence between current and previous is 0.343587 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_ad_reward'                                                   The approximate Jensen-Shannon divergence between current and previous is 0.672075 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_use_extra_steps'                                             The approximate Jensen-Shannon divergence between current and previous is 0.390741 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_post_score'                                                  The approximate Jensen-Shannon divergence between current and previous is 0.352555 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_level_complete_quickplay'                                     The approximate Jensen-Shannon divergence between current and previous is 0.48113 (up to six significant digits), above the threshold 0.3.  \n",
       "'country'                       The Linfty distance between current and previous is 0.0168421 (up to six significant digits), above the threshold 0.003. The feature value with maximum difference is: Taiwan  \n",
       "'cnt_spend_virtual_currency'                                      The approximate Jensen-Shannon divergence between current and previous is 0.332597 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_challenge_a_friend'                                          The approximate Jensen-Shannon divergence between current and previous is 0.737841 (up to six significant digits), above the threshold 0.3.  \n",
       "'cnt_level_start_quickplay'                                       The approximate Jensen-Shannon divergence between current and previous is 0.363518 (up to six significant digits), above the threshold 0.3.  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    my_model_monitor.show_feature_drift_stats(model_monitoring_job_1.name)\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "qtjLw8rVcC13",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No stats were generated.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    my_model_monitor.show_feature_drift_stats(model_monitoring_job_2.name)\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "-mxYA55nz6tO",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No stats were generated.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/8743820040400273408 current state:\n",
      "JobState.JOB_STATE_RUNNING\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitoringJob projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384/modelMonitoringJobs/3436327879544143872 current state:\n",
      "JobState.JOB_STATE_RUNNING\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    my_model_monitor.show_output_drift_stats(model_monitoring_job_2.name)\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hovSbsmBmEKZ"
   },
   "source": [
    "### Step 6: Clean Up\n",
    "\n",
    "If you no longer need your model monitoring resources, run the following to delete them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "ejHg5PqiJPS2",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Deleting ModelMonitor : projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:Delete ModelMonitor  backing LRO: projects/684754974539/locations/us-central1/operations/6876846358678994944\n",
      "INFO:vertexai.resources.preview.ml_monitoring.model_monitors:ModelMonitor deleted. . Resource name: projects/684754974539/locations/us-central1/modelMonitors/2411541261015056384\n"
     ]
    }
   ],
   "source": [
    "# Delete the model monitor\n",
    "my_model_monitor.delete(force=True)\n",
    "\n",
    "# Delete the model\n",
    "model.delete()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "model_monitoring_for_custom_model_batch_prediction_job.ipynb",
   "provenance": []
  },
  "environment": {
   "kernel": "conda-base-py",
   "name": "workbench-notebooks.m134",
   "type": "gcloud",
   "uri": "us-docker.pkg.dev/deeplearning-platform-release/gcr.io/workbench-notebooks:m134"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel) (Local)",
   "language": "python",
   "name": "conda-base-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
